fix: close batch execution and recovery lifecycle

This commit is contained in:
阿白叔 committed 2026-11-17 16:56:29 +08:00
1 parent b19afc6a53
commit d9cf6a07e9
91 files changed
+5779 -454

No files matched your search

@@ -0,0 +1,32 @@
# 批量替换阻塞与状态闭环实施计划
按用户在本聊天确认的设计执行。保留此前 issue29 修复,不修改其他聊天的 independent-verification 文档。
## 用户已确定的规则
- 位号可明确定位时忽略 UID 缺失、异常、重复或变化;贯通清单、计划、执行、回读、恢复和网表。
- 同工程统一保存/关闭重开/回读一次,暂停时提前提交;提交前不得记作 saved_ok。
- Mentor 和全部自有写进程确认退出后,隔离残留桌面并放行新批次,不假报桌面已释放。
- 结果不确定的工程保留待核验并隔离,同写入源一并隔离;释放全部自有写进程后继续独立工程。人工暂停停止整批。
- 操作回执、任务、队列状态幂等收敛。已保存结果、网表验收、资源告警分别展示。
- 补齐关键调用进入/返回和等待原因及诊断包,不让清理错误覆盖原错误。
## 实施任务
1. 位号身份:原生清单与目标键使用位号身份,保留原始 UID 证据;旧计划/成功操作编号不变,执行与网表允许位号关联,合法多单元及真实歧义分别处理。
2. 工程提交/暂停:增加待保存及工程提交记录,工程结束或暂停统一保存回读,所有待提交项原子转完成;SafeStop 不包装成 Baseline,确认未写入暂停返回 pending/paused;未知保存仍 blocked。
3. 资源隔离:新增隔离终态和明确阻断字段,持久化进程退出证明;残留桌面不重复 Quit/CloseDesktop,新会话可进入;未知写进程不得隔离放行。
4. 工程与任务隔离:新增 BLOCKED,工程异常释放后继续独立工程,区分本地回收控制和用户暂停;共享工程/iCDB写入源隔离,旧回执不重放。
5. 诊断与状态:明确停止原因/等待原因,记录关键操作进入返回;诊断包含执行日志、worker进度、原生报告和提交证据,保留脱敏/体积限制;UI正确显示 blocked/failed/paused/completed。
6. 集成验收:按设计20种场景测试,回归现有队列/恢复/资源/UI/网表;本机CAD仅副本,不把fake和CLI协议验证称为真实CAD验收;记录每工程save次数及阶段计时。打包候选版但不自动发布旧Release。
## 接口约定
- 现有 @REFDES| 标识复用,不改写旧成功操作编号。
- saved_ok 仅表示保存回读已通过。新增暂存状态必须由工程提交记录覆盖后才能转 verified。
- 隔离桌面保持 desktop_released=false;写进程退出证据与桌面存在性分开。
- BLOCKED是业务待核验,不等于全局资源未释放。
## 工作区与验证
当前分支 codex/issue29-blocking-cleanup,base b19afc6。已有未提交变更已完成此前回归,用户授权本轮继续实施。每个非平凡行为先RED后GREEN。阶段回归按实际修改选择,最终检查 tracked diff、打包产物和验收覆盖。
@@ -0,0 +1,109 @@
# 批量执行闭环验收
本轮对应 `docs/superpowers/plans/2026-11-17-batch-execution-closure.md`。位号优先定位;工程内先暂存,工程统一保存回读后提交;不确定工程隔离,确认全部自有写进程退出后继续独立工程。工程执行、网表验收、资源告警分开记录。
```mermaid
flowchart TD
A[导入 BOM 并保存待执行计划] --> B[原生读取当前工程所需位号与网表]
B --> C{工程/页/位号能否明确定位}
C -- 否 --> D[记录明确的匹配错误]
C -- 是 --> E[逐项原生替换和内存核验]
E --> F[写 pending_save / 暂存回执]
F --> G{工程还有项且未暂停}
G -- 是 --> E
G -- 否 --> H[统一保存页 / 关闭工程 / 重开回读 / 最终关闭]
H --> I{全部前缀核验通过}
I -- 是 --> J[原子提交已保存回执和任务进度]
J --> K{人工暂停}
K -- 是 --> L[结束整批 / 保留未执行项]
K -- 否 --> M[继续下一独立工程]
E -- 未知写入或异常 --> N[保留 uncertain / 工程待核验]
I -- 否 --> N
N --> O[回收该工程自有写进程]
O --> P{写进程退出和自有句柄关闭已证实}
P -- 否 --> Q[停止整批 / 等待资源核验]
P -- 是 --> R[隔离工程及共享写入源]
R --> S{旧桌面是否仍存在}
S -- 是 --> T[保留隔离告警 / desktop_released=false]
S -- 否 --> M
T --> M
J --> U[独立原生网表验收 / 单独显示结果]
R --> V[共享源后续任务未派发即阻塞]
V --> W[原依赖恢复后显式核验重新准入]
```
## 20 类验收场景
下面是自动化验收矩阵,不代表 20 次真实 Mentor 替换。Python 测试替代 COM 边界;VBS 测试执行生产脚本函数并使用假 COM 对象。真实 Windows 桌面测试另列。
| # | 场景 | 需要确认的结果 | 自动化覆盖 |
|---|---|---|---|
| 1 | 器件 UID 缺失或格式异常 | 位号明确时可定位,原始 UID 留证 | `test_refdes_identity_closure` inventory cases |
| 2 | 不同位号共享 UID | 不误绑定,不阻断无关器件 | 同模块 duplicate UID / unselected collision cases |
| 3 | 替换或重开后 UID 变化 | 快照、回读、网表仍关联原位号,旧操作编号不变 | 同模块 snapshot/readback/netlist cases |
| 4 | 原 UID 被另一位号复用 | 不把错误位号当成替换成功 | 同模块 reused UID mismatch cases |
| 5 | 不同页或层级同位号 | 依据工程、页和层级定位,真实歧义拒绝 | 同模块 sheet/hierarchy cases |
| 6 | 同一物理器件多个合法单元 | 全单元快照及回读,物理引脚连接仍严格比较 | 同模块 multiunit cases |
| 7 | 工程中多项暂存 | 游标推进,但 saved/completed 不提前增加 | `test_project_commit_progress` |
| 8 | 工程末统一保存 | 每个已修改页保存一次,工程关闭重开回读一次 | `test_project_commit_vbs` save/reopen case |
| 9 | 暂存前缀后人工暂停 | 提交已修改前缀,未执行项不产生写入/不确定状态 | `test_project_commit_end_to_end` pause cases |
| 10 | 写入前暂停 | 保留 SafeStop 来源,当前项保持 pending | `test_project_commit_vbs` / `test_resource_isolation_prewrite_pause` |
| 11 | 回读失败或提交清单不完整 | 不提前提交任何暂存项 | `test_project_commit_journal` / `test_project_commit_proof` |
| 12 | 保存成功后程序崩溃 | 从不可变提交证据幂等恢复,不重放替换 | `test_project_commit_journal` / `test_project_commit_public_recovery` |
| 13 | 工程保存回读持续有进度且超过原等待时间 | 不因固定 60 秒误判收尾失败 | `test_project_commit_session` progressing commit case |
| 14 | 自有进程退出但旧桌面仍被外部句柄持有 | 保留 desktop_released=false,隔离告警,新批次可进入 | `test_resource_isolation`;真实 Windows 探针 |
| 15 | 仍有写进程或进程身份未知 | 不允许隔离放行 | `test_resource_isolation` active/unknown cases |
| 16 | 隔离后重启,随后外部句柄消失 | 重验退出证明;被动确认释放,不重复 Quit | 同模块 restart/passive disappearance cases |
| 17 | 工程结果未知,资源已收尾 | 保留 blocked 和 uncertain,继续独立工程,恢复队列不重放 | `test_project_isolation_queue` / `test_workspace_execution` |
| 18 | 本地回收与人工停止 | 本地回收不污染下一工程;人工暂停/强制结束仍生效 | `test_project_isolation_queue` / workspace control cases |
| 19 | 后续工程共享 iCDB 写入源 | 未派发即阻塞,原依赖核验完成后才重新准入 | `test_lifecycle_integration_fix` dependency cases |
| 20 | 任务完成、失败、恢复及资源告警 | 实时分类;已保存结果与网表/资源结果分别展示;诊断含最后成功项和未返回调用 | `test_live_task_completion` / `test_failed_native_recovery_guard` / `test_diagnostics_closure` |
## 证据位置
- 本轮回归日志:`build/batch-execution-closure/`。
- Windows 桌面实际持有、重启复核及新桌面准入:`build/batch-execution-closure/resource-isolation/run-4ace6aa7a36b4204a7bed568b5ce7505/evidence.json`。
- 工单原始日志保留在 `outputs/issues-20261117/issue29/`,不修改用户上传证据。
## 自动化验证结果
主程序/UI 全量运行 2335 项,最初 2320 项通过、15 项失败。14 项仍断言“未知结果必须整批暂停”,已按已批准的工程隔离策略更新,同时保留同工程后项不派发、不重放和独立工程可继续的断言;另一项修复测试对尚未创建完成的原生弹窗按钮的读取竞态。随后对全部失败项和本轮新增/修改的调用链定点复验。
最终 JUnit 合并记录为 **2438 个唯一用例通过,0 个未解决失败**,证据见 `build/batch-execution-closure/verified-test-summary.json`。这是全量运行加定点复验的合并结果,不是另一次全部用例在最后源码上的全量运行。定点记录为 root delta 143 项、lifecycle delta 97 项、project recovery 29 项、identity delta 47 项、deleted guard 1 项,均通过;这些记录存在重叠,不应直接相加。
独立审查已复核位号/多单元、工程暂存及暂停、资源隔离、共享源恢复及公开工程提交恢复。工程已保存但数据库回执未完成时,公开恢复入口验证原生关闭检查点、保存源、回读文件及原尝试身份后恢复回执,不启动 CAD 或重复替换;源变化、其他未知写入、证据损坏及不匹配的中断记录继续保持待核验。
## 实机边界
工程级批量保存、UID 异常和多单元替换需要使用副本进行真实 Mentor 验收。自动化协议验证、真实桌面句柄探针和真实 CAD 写入必须分别报告;不得用前两者声称已完成原 100 工程队列。原工程、中心库、Excel 和历史 uncertain 记录不作为试验修改对象。
首轮真实 Mentor 探针 `build/batch-execution-closure/mentor-live/` 在 RL02 原生替换和内存回读后失败:新暂存入口未暂时清除内存 `restored_ok`,导致生产命令读取器认定没有待处理命令。该问题已有真实读取器回归;复测使用 `mentor-live-fixed/` 全新副本。失败副本及 uncertain 保留,不重放。首轮 Mentor 正常退出、桌面释放,无强制终止;原备份 34 个文件前后哈希一致。
第二轮 `mentor-live-fixed/` 发现未选中非 PCB 图形的空位号被写成相同别名,导致后续全页原生回读重复 UID;已对空位号跳过引用别名并保留其独立原始 UID。第三轮 `mentor-live-final/` 通过该位置并写出 `pending_save`,随后全页语义采集缺少逐器件进度,触发无进度停止;补充实际完成计数后使用 `mentor-live-progress/` 新副本复测。前三轮异常资源均确认释放,备份源文件前后哈希一致;不计为工程保存成功验收。
第四轮真实 Mentor 验收通过,证据为 `build/batch-execution-closure/mentor-live-progress/closure-live-summary.json`。RL02、RL05 两项替换先暂存,最后统一提交;日志确认一个 worker、一次保存、两次打开工程(首次打开及保存后重开)。两项 SQL 操作均为 `verified/generation=1`,工程提交 `verified`,任务 `completed`。独立原生清单读取复核 13 个位号:两项目标料号正确,其余 11 项保留原料号。Mentor 和桌面确认释放,无强制终止;备份源 34 个文件哈希未变化。
### 本机计时及限制
| 实际边界 | 秒 |
|---|---:|
| 第一项开始至暂存回执 | 472.664 |
| 第二项派发至暂存回执,含前一项检查点核验 | 539.298 |
| 工程收尾,含完整状态检查、统一保存、关闭重开和回读 | 431.863 |
| 其中两次原生替换提交调用 | 0.631 / 0.424 |
| Mentor 退出 | 5.093 |
| 桌面释放 | 0.009 |
| 整个队列 | 1450.341 |
| 队列结束后的独立原生清单复核 | 1.505 |
“其中”调用嵌套于上层步骤,不能再次相加。本轮同时存在构建和回归负载,不能作为独占性能基准,也没有同条件旧版对照。全页状态反复采集仍是明显耗时项;本次证明阻塞闭环、保存次数与结果提交正确,**没有证明速度提升**。真实 CAD 仅验证一个副本工程、两次替换;多单元、UID 异常和 20 类异常场景主要依靠自动化协议/COM 边界测试,不能当成 20 个真实工程已通过。
候选包仅供本地测试,不自动覆盖旧 Release。
## 本地候选程序
入口:`D:\CodexValidation\batch-closure-20261117\candidate\dist\DxPartReplace\DxPartReplace.exe`。这是包含完整 Harness/Node/Qt 运行时的目录包,运行时文件须与程序一起保留。应用版本资源仍为 1.3.1,本轮候选代码通过冻结模块核对区分,不代表远端已经发布了新版本。
冻结包内 162 个应用模块与最终源码编译结果一致,两份 VBS 与源码逐字节一致。EXE SHA256:`4b19948b6a008912851f3ab674afa361bf4a28de4b74f20d3d04b80719200cc1`。打包程序本身已通过启动、冻结监督进程、进程身份读取、资源释放及正常退出验证。报告位于候选目录上级的 `source-verification.json` 与 `packaged-runtime-verification.json`。
实施决定:沿用已有功能分支保留先前授权修复;互不重叠的模块并行处理后统一审查;本地直接交付完整目录包。前两项若判断错误需集成返工,目录包如需转移到其他机器可另行归档。本轮未合并、推送或发布远端版本。
@@ -0,0 +1,71 @@
{
"verification_method": "Full main/UI run plus focused reruns for amended behavior; not a second whole-suite run.",
"unique_verified_cases": 2438,
"unresolved_failures": 0,
"junit_sha256": {
"final-regression.xml": "3e06b8c07bb057a039d67ce62eb2dcc133a52c0a24af1158c0f32cb8658dc277",
"root-delta.xml": "9124ddb63d063f4c07319e83541d4b8da2dfe0376d03ba4307568e8631d98f00",
"lifecycle-delta.xml": "1a753e128f82aad2c93ac6a56ff02201141bb7a2df178066d4376d8252a4cdb5",
"project-recovery-delta.xml": "a6834285d840263bcddd908d86594c9f742f696490a52bb2b2b382e4c7ffecaf",
"identity-final-delta.xml": "7990ce9459db3dec29c3ddbec3b6080569f5c527f3240bc8d896bb659e34329c",
"lifecycle-deleted-delta.xml": "fbdc727e207b747f5d4f95db437e04f573572fe74828d9d85dc9f3c6ad1fc445"
},
"real_mentor": {
"verified": true,
"real_mentor": true,
"targets": [
"RL02",
"RL05"
],
"native_inventory_readback_targets": 13,
"source_unchanged": true,
"source_files": 34,
"save_calls": 1,
"project_open_calls": 2,
"worker_starts": 1,
"saved_items": 2,
"operations": [
[
"verified",
1
],
[
"verified",
1
]
],
"project_commits": [
[
"verified"
]
],
"queue_seconds": 1450.341202999989,
"independent_readback_seconds": 1.5047315999981947,
"resource_state": "released",
"forced_termination": false,
"limitation": "Two replacement items on one copied project; concurrent tests/build affected timing; no speedup claim."
},
"review_reports": [
"identity-review.md",
"progress-review.md",
"lifecycle-review.md",
"final-review.md"
],
"candidate_published": false,
"packaged_source_verification": {
"passed": true,
"compiled_application_modules": 162,
"mismatches": [],
"executable_sha256": "4b19948b6a008912851f3ab674afa361bf4a28de4b74f20d3d04b80719200cc1",
"full_harness_present": true
},
"packaged_runtime_verification": {
"passed": true,
"frozen_supervisor": true,
"win32timezone_archived": true,
"process_creation_time_read": true,
"resources_released": true,
"executable": "D:\\CodexValidation\\batch-closure-20261117\\candidate\\dist\\DxPartReplace\\DxPartReplace.exe",
"data_root": "D:\\CodexValidation\\batch-closure-20261117\\candidate\\runtime-smoke"
}
}
@@ -249,6 +249,7 @@ Function QPNativeCollectMode(ByRef SelectedUids, ByVal Token, ByVal Project, ByV
QPNativeRequireContext Project, Sheet, Guid
QPInvalidatePinIndex
Set Components = QP_App.ActiveView.Query(VDM_COMP, VD_ALL)
QPBindSheetReferences Components
Set Seen = CreateObject("Scripting.Dictionary")
Set Found = CreateObject("Scripting.Dictionary")
Text = "NR1" & vbTab & Token & vbTab & QPEncode(Project) & vbTab & QPEncode(Sheet) & vbCrLf
@@ -306,6 +307,13 @@ Function QPBuildNativeReadScript(ByVal Source, ByRef SelectedUids, ByVal Token,
For Each Uid In SelectedUids.Keys
Body = Body & "QP_NR_Selected.Add " & QPNativeExpression(Uid) & ",True" & vbCrLf
Next
If IsObject(QP_ReferenceAliases) Then
Body=Body & "Set QP_ReferenceAliases=CreateObject(""Scripting.Dictionary"")" & vbCrLf
For Each Uid In QP_ReferenceAliases.Keys
Body=Body & "QP_ReferenceAliases.Add " & QPNativeExpression(Uid) & "," & _
QPNativeExpression(QP_ReferenceAliases(Uid)) & vbCrLf
Next
End If
Body = Body & "On Error Resume Next" & vbCrLf & _
"Set QP_App=GetObject(,""Viewdraw.Application"")" & vbCrLf & _
"If Err.Number=0 Then QP_NR_Text=QPNativeCollectMode(QP_NR_Selected," & _
@@ -315,7 +323,7 @@ Function QPBuildNativeReadScript(ByVal Source, ByRef SelectedUids, ByVal Token,
"If QP_NR_Code<>0 Then QP_NR_Text=""ERROR"" & vbTab & " & QPNativeExpression(Token) & _
" & vbTab & CStr(QP_NR_Code) & vbTab & QPEncode(QP_NR_Source) & vbTab & QPEncode(QP_NR_Description) & vbCrLf" & vbCrLf & _
"QPWriteUtf8 " & QPNativeExpression(ResultPath) & ",QP_NR_Text" & vbCrLf & "On Error GoTo 0" & vbCrLf
Names = Split("QPNativeCollectMode QPNativeRequireContext QPNativeReadInvalid QPSnapshotComponent QPUntouchedComponent QPUntouchedFullState QPUntouchedIdentity QPComponentPins QPComponentIdentityKey QPInvalidatePinIndex QPConnectionSignatureFromPins QPAddSortedSignature QPSnapshotAttribute QPReadOptionalAttributeValue QPOptionalAttributeField QPSortSnapshotLines QPLog QPAppendAscii QPEncode QPDecode QPProjectPath QPSheetIdentity QPCanonicalPath QPEnsureProjectAccess QPReadUtf8 QPWriteUtf8", " ")
Names = Split("QPNativeCollectMode QPNativeRequireContext QPNativeReadInvalid QPSnapshotComponent QPUntouchedComponent QPUntouchedFullState QPUntouchedIdentity QPComponentPins QPComponentIdentityKey QPComponentUnitKey QPBindSheetReferences QPInvalidatePinIndex QPConnectionSignatureFromPins QPAddSortedSignature QPSnapshotAttribute QPReadOptionalAttributeValue QPOptionalAttributeField QPSortSnapshotLines QPLog QPAppendAscii QPEncode QPDecode QPProjectPath QPSheetIdentity QPCanonicalPath QPEnsureProjectAccess QPReadUtf8 QPWriteUtf8", " ")
Pattern.Global = False
For Each Name In Names
Pattern.Pattern = "^(Function|Sub) " & Name & "\([^\n]*\n[\s\S]*?^End \1[ \t]*"
+365 -33
View File
@@ -28,17 +28,20 @@ Dim QP_PinContext
Dim QP_ComponentUidIndex, QP_ComponentRefdesIndex, QP_ComponentLocationIndex
Dim QP_FallbackIdentityUsed
Dim QP_FallbackIdentityWarnings
Dim QP_ReferenceAliases
Dim QP_ComponentIndexContext
Dim QP_NativeCallActive
Dim QP_BatchMode, QP_OwnedProject, QP_BatchState, QP_BatchCode, QP_BatchMessage
Dim QP_BatchSaveCommitted, QP_BatchSaveRequested
Dim QP_DiscoveryMode, QP_TargetSchematic, QP_TargetSheet
Dim QP_FailureNumber, QP_FailureSource, QP_FailureDescription
Dim QP_FailureCleanupActive
Dim QP_NcSyncSummary
Dim QP_SessionMode, QP_SessionToken, QP_SessionProject, QP_MonitorJobId
Dim QP_OwnedProjectPath, QP_SessionWaiting, QP_SessionExternalChange
Dim QP_CheckpointSchematic, QP_CheckpointSheet
Dim QP_ProjectAccess, QP_AccessProject
Dim QP_SessionItems, QP_SessionSheets, QP_SessionDocuments, QP_SessionCommitActive
Dim FailureState, CloseError
Set QP_Fso = CreateObject("Scripting.FileSystemObject")
QP_RootTemplate = "%LOCALAPPDATA%\DxProductivity\trial-task-center\quick-run"
@@ -65,6 +68,7 @@ QP_OwnedProject = False
QP_BatchState = ""
QP_BatchSaveCommitted = False
QP_BatchSaveRequested = False
QP_FailureCleanupActive = False
QP_DiscoveryMode = False
QP_TargetSchematic = ""
QP_TargetSheet = ""
@@ -82,6 +86,10 @@ QP_SessionMode = False
QP_OwnedProjectPath = ""
QP_SessionWaiting = False
QP_SessionExternalChange = False
Set QP_SessionItems = CreateObject("Scripting.Dictionary")
Set QP_SessionSheets = CreateObject("Scripting.Dictionary")
Set QP_SessionDocuments = CreateObject("Scripting.Dictionary")
QP_SessionCommitActive = False
QP_MonitorJobId = QP_ExpectedJobId
Set QP_App = Nothing
@@ -91,6 +99,7 @@ If Err.Number <> 0 Or QP_App Is Nothing Then
QP_FailureNumber = Err.Number
QP_FailureSource = Err.Source
QP_FailureDescription = Err.Description
If QP_FailureSource = "QuickPartReplace.SafeStop" And QP_SessionMode Then QPAcknowledgePrewritePause
QP_ErrorMessage = "stage=" & QP_Stage & " " & CStr(QP_FailureSource) & ": " & CStr(QP_FailureDescription)
If QP_DiscoveryMode Then QPCleanupDiscoveryFailure
QPFail QP_Root, "hard_failure", QP_FailureNumber, QP_FailureSource, QP_FailureDescription
@@ -115,6 +124,7 @@ Else
QP_FailureNumber = Err.Number
QP_FailureSource = Err.Source
QP_FailureDescription = Err.Description
If QP_FailureSource = "QuickPartReplace.SafeStop" And QP_SessionMode Then QPAcknowledgePrewritePause
If QP_DiscoveryMode Then QPCleanupDiscoveryFailure
FailureState = "hard_failure"
If QP_BatchSaveRequested Then
@@ -161,7 +171,7 @@ Sub QPMain()
Set Command = QPReadCommand(QP_Fso.BuildPath(QP_Root, "command.tsv"))
QP_CurrentJobId = CStr(Command("job_id"))
QPRequireProject Command
If QP_BatchMode And Not QP_Fso.FileExists(QP_Fso.BuildPath(QP_Root, "recovery.request")) Then _
If QP_BatchMode And QP_SessionItems.Count = 0 And Not QP_Fso.FileExists(QP_Fso.BuildPath(QP_Root, "recovery.request")) Then _
QPVerifyExpectedBaselines CStr(Command("project_path"))
SnapshotPath = QPJobSnapshotPath(CStr(Command("job_id")))
State = QPReadState(QP_Fso.BuildPath(QP_Root, "status.tsv"), _
@@ -187,7 +197,7 @@ Sub QPMain()
QPCaptureAndSelect Command, SnapshotPath
If QPReadState(QP_Fso.BuildPath(QP_Root, "status.tsv"), QP_CurrentJobId) = "no_match" Then Exit Sub
QPAutomaticReplaceAndRestore Command, SnapshotPath
ElseIf State = "awaiting_native_replace" Then
ElseIf State = "awaiting_native_replace" Or State = "pending" Then
QPAutomaticReplaceAndRestore Command, SnapshotPath
Else
Err.Raise vbObjectError + 796, "QuickPartReplace", _
@@ -266,6 +276,9 @@ Sub QPProgress(ByVal Stage, ByVal Done, ByVal Total)
Safe = 0
If Not CBool(QP_NativeMutationPossible) And Not CBool(QP_RestoreTransactionOpen) And _
Not CBool(QP_NativeCallActive) Then Safe = 1
If QP_SessionMode And IsObject(QP_SessionItems) Then
If QP_SessionItems.Exists(QP_CurrentJobId) Then Safe = 0
End If
QP_ProgressSequence = CLng(QP_ProgressSequence) + 1
Job = QP_ExpectedJobId
Suffix = ""
@@ -276,7 +289,7 @@ Sub QPProgress(ByVal Stage, ByVal Done, ByVal Total)
WScript.StdOut.WriteLine "DX_PROGRESS" & vbTab & QP_AttemptToken & vbTab & _
Job & vbTab & CStr(QP_ProgressSequence) & vbTab & Stage & _
vbTab & CStr(Done) & vbTab & CStr(Total) & vbTab & CStr(Safe) & Suffix
If Safe = 1 And Stage <> "finished" Then
If Safe = 1 And Stage <> "finished" And Stage <> "session_wait" And Not QP_FailureCleanupActive Then
StopPath = QP_Fso.BuildPath(QP_Root, "monitor-" & QP_AttemptToken & ".stop")
If QP_Fso.FileExists(StopPath) Then
If Trim(QPReadUtf8(StopPath)) = CStr(QP_AttemptToken) Then
@@ -295,6 +308,41 @@ Sub QPProgress(ByVal Stage, ByVal Done, ByVal Total)
End If
End Sub
Sub QPAcknowledgePrewritePause()
Dim Command, PauseText, ErrorNumber, ErrorDescription
If QP_NativeMutationPossible Or QP_RestoreTransactionOpen Or QP_NativeCallActive Or _
QP_BatchSaveRequested Or QP_SessionCommitActive Or QP_SessionItems.Exists(QP_CurrentJobId) Then Exit Sub
QP_FailureCleanupActive = True
On Error Resume Next
Err.Clear
If QP_OwnedProject Then
QP_SessionWaiting = False
QPVerifyFailedOriginalSnapshot
If Err.Number = 0 Then
Set Command = QPReadCommand(QP_Fso.BuildPath(QP_Root, "command.tsv"))
If Err.Number = 0 Then QPWriteVerifiedState Command
End If
If Err.Number = 0 Then QPCommitSessionProject
End If
ErrorNumber = Err.Number
ErrorDescription = Err.Description
Err.Clear
If ErrorNumber = 0 And Not QP_OwnedProject Then
PauseText = QP_SessionToken & vbTab & QPEncode(QP_SessionProject) & vbTab & QP_ExpectedJobId & vbTab & "prewrite_closed"
QPWriteUtf8 QP_Fso.BuildPath(QP_Root, "project-pause-" & QP_SessionToken & ".tsv"), PauseText
If Err.Number = 0 Then
WScript.StdOut.WriteLine "DX_PREWRITE_PAUSED" & vbTab & QP_AttemptToken & vbTab & _
QP_MonitorJobId & vbTab & QP_SessionToken & vbTab & QP_ExpectedJobId & vbTab & CStr(QP_ProgressSequence)
WScript.Quit 75
End If
ErrorDescription = Err.Description
End If
Err.Clear
On Error GoTo 0
QP_FailureCleanupActive = False
QP_FailureDescription = QP_FailureDescription & "; prewrite pause closure was not proven: " & ErrorDescription
End Sub
Sub QPReadSessionRequest()
Dim Path, Fields, Index, Character
Path = QP_Fso.BuildPath(QP_Root, "project-session.request")
@@ -331,7 +379,7 @@ Sub QPSessionNext()
QPRequireSessionCheckpoint
QP_SessionWaiting = False
If Fields(1) = "close" Then
QPCloseBatchProject QP_SessionProject, False
QPCommitSessionProject
QP_SessionMode = False
Exit Sub
End If
@@ -347,15 +395,6 @@ Sub QPSessionNext()
TargetPath = QPConfirmedProjectPath()
If QPCanonicalPath(TargetPath) <> QPCanonicalPath(QP_SessionProject) Then _
Err.Raise vbObjectError + 873, "QuickPartReplace.ProjectSession", "cannot switch project inside a session"
Set View = QP_App.ActiveView
If Not View Is Nothing Then
If CStr(View.GetName(0)) <> QP_TargetSchematic Or CStr(View.Block.SheetNum) <> QP_TargetSheet Then
Set Documents = QP_App.SchematicSheetDocuments()
If Documents.Count <> 1 Then Err.Raise vbObjectError + 873, "QuickPartReplace.ProjectSession", "owned sheet set changed"
Set Document = Documents.Item(1)
Document.Close False, ""
End If
End If
QPEnsureProject QP_App, TargetPath
Exit Sub
End If
@@ -395,10 +434,12 @@ Sub QPRequireSessionCheckpoint()
Err.Raise vbObjectError + 873, "QuickPartReplace.ProjectSession", "project modified outside worker at checkpoint; no automatic save or close"
Set Documents = QP_App.SchematicSheetDocuments()
Set View = QP_App.ActiveView
If Documents.Count <> 1 Or View Is Nothing Then _
If Documents.Count <> QP_SessionDocuments.Count Or View Is Nothing Then _
Err.Raise vbObjectError + 873, "QuickPartReplace.ProjectSession", "owned sheet set changed at checkpoint"
If CStr(View.GetName(0)) <> QP_CheckpointSchematic Or CStr(View.Block.SheetNum) <> QP_CheckpointSheet Then _
Err.Raise vbObjectError + 873, "QuickPartReplace.ProjectSession", "active sheet changed at checkpoint"
If QP_SessionSheets.Exists(QPSheetKey(QP_CheckpointSchematic, QP_CheckpointSheet)) Then _
QPRequireExpectedState QP_SessionSheets.Item(QPSheetKey(QP_CheckpointSchematic, QP_CheckpointSheet)), QPCurrentSemanticState()
End Sub
Sub QPInvalidatePinIndex()
@@ -691,6 +732,10 @@ Sub QPEnsureProject(ByRef App, ByVal TargetPath)
Set Document = Documents.Open(OnlySchematic, OnlySheet)
If Document Is Nothing Then Err.Raise vbObjectError + 857, "QuickPartReplace.Session", "target sheet could not be opened"
Document.Activate
If QP_SessionMode Then
If Not QP_SessionDocuments.Exists(QPSheetKey(OnlySchematic, OnlySheet)) Then _
QP_SessionDocuments.Add QPSheetKey(OnlySchematic, OnlySheet), Document
End If
Set View = App.ActiveView
If View Is Nothing Then Err.Raise vbObjectError + 857, "QuickPartReplace.Session", "target sheet did not become active"
If CStr(View.GetName(0)) <> OnlySchematic Or CStr(View.Block.SheetNum) <> OnlySheet Then _
@@ -806,7 +851,7 @@ Sub QPDiscoverInventory(ByRef Documents)
End Sub
Function QPComponentIdentityKey(ByRef Comp)
Dim Uid, Refdes, Number, Description
Dim Uid, Refdes, Number, Description, AliasKey, UnitKey
On Error Resume Next
Err.Clear
Uid = CStr(Comp.UID)
@@ -814,8 +859,7 @@ Function QPComponentIdentityKey(ByRef Comp)
Description = Err.Description
Err.Clear
On Error GoTo 0
If Number <> 0 Then Err.Raise vbObjectError + 872, "QuickPartReplace.Discovery", _
"component UID could not be read: " & Description
If Number <> 0 Then Uid = ""
On Error Resume Next
Err.Clear
Refdes = CStr(Comp.Refdes)
@@ -825,6 +869,20 @@ Function QPComponentIdentityKey(ByRef Comp)
On Error GoTo 0
If Number <> 0 Then Err.Raise vbObjectError + 872, _
"QuickPartReplace.Discovery", "component Refdes is unreadable; operation stopped before any replacement"
If Len(Trim(Refdes))>0 And IsObject(QP_ReferenceAliases) Then
AliasKey = LCase(QPSheetIdentity() & "|" & Trim(Refdes))
If QP_ReferenceAliases.Exists(AliasKey & "|#units") Then
UnitKey = AliasKey & "|unit=" & QPComponentUnitKey(Comp)
If Not QP_ReferenceAliases.Exists(UnitKey) Then Err.Raise vbObjectError + 872, _
"QuickPartReplace.Identity", "physical unit changed or is ambiguous"
QPComponentIdentityKey = QP_ReferenceAliases(UnitKey)
Exit Function
End If
If QP_ReferenceAliases.Exists(AliasKey) Then
QPComponentIdentityKey = QP_ReferenceAliases(AliasKey)
Exit Function
End If
End If
If Len(Trim(CStr(Uid))) > 0 Then
QPComponentIdentityKey = Trim(CStr(Uid))
Exit Function
@@ -844,6 +902,89 @@ Function QPComponentIdentityKey(ByRef Comp)
End If
End Function
Sub QPBindSheetReferences(ByRef Components)
Dim Counts, References, Units, Comp, Uid, Refdes, Key, Context, UnitKey, Identity
Set Counts=CreateObject("Scripting.Dictionary")
Set References=CreateObject("Scripting.Dictionary")
Set Units=CreateObject("Scripting.Dictionary")
Context=LCase(QPSheetIdentity())
If Not IsObject(QP_ReferenceAliases) Then Set QP_ReferenceAliases=CreateObject("Scripting.Dictionary")
For Each Comp In Components
Uid=""
On Error Resume Next
Err.Clear
Uid=Trim(CStr(Comp.UID))
Err.Clear
On Error GoTo 0
If Counts.Exists(Uid) Then Counts(Uid)=Counts(Uid)+1 Else Counts.Add Uid,1
Refdes=LCase(Trim(CStr(Comp.Refdes)))
If References.Exists(Refdes) Then References(Refdes)=References(Refdes)+1 Else References.Add Refdes,1
Next
For Each Comp In Components
Uid=""
On Error Resume Next
Err.Clear
Uid=Trim(CStr(Comp.UID))
Err.Clear
On Error GoTo 0
Refdes=Trim(CStr(Comp.Refdes))
Key=Context & "|" & LCase(Refdes)
If Len(Refdes)>0 Then
Identity=Uid
If Len(Uid)=0 Or Counts(Uid)>1 Then Identity="@REFDES|" & Refdes
If References(LCase(Refdes))>1 Then
UnitKey=Key & "|unit=" & QPComponentUnitKey(Comp)
If Units.Exists(UnitKey) Then Err.Raise vbObjectError+872, _
"QuickPartReplace.Identity","physical unit association is ambiguous"
Units.Add UnitKey,True
If Len(Uid)=0 Or Counts(Uid)>1 Then Identity=Identity & "|UNIT|" & QPComponentUnitKey(Comp)
If Not QP_ReferenceAliases.Exists(Key & "|#units") Then QP_ReferenceAliases.Add Key & "|#units","1"
If Not QP_ReferenceAliases.Exists(UnitKey) Then
QP_ReferenceAliases.Add UnitKey,Identity
End If
ElseIf Not QP_ReferenceAliases.Exists(Key) Then
QP_ReferenceAliases.Add Key,Identity
End If
End If
Next
End Sub
Function QPComponentUnitKey(ByRef Comp)
Dim Point
Set Point=Comp.GetLocation()
QPComponentUnitKey=CStr(Point.X) & "|" & CStr(Point.Y) & "|" & CStr(Comp.Orientation) & "|" & CStr(Comp.Scale)
End Function
Sub QPBindSnapshotReference(ByVal Context, ByRef Record)
Dim Key, UnitKey
If Len(Trim(CStr(Record("refdes"))))=0 Then Exit Sub
If Not IsObject(QP_ReferenceAliases) Then Set QP_ReferenceAliases=CreateObject("Scripting.Dictionary")
Key=LCase(Context & "|" & Trim(Record("refdes")))
UnitKey=Key & "|unit=" & Record("x") & "|" & Record("y") & "|" & Record("orientation") & "|" & Record("scale")
If QP_ReferenceAliases.Exists(Key) Then
If QP_ReferenceAliases(Key)<>Record("uid") Then QP_ReferenceAliases(Key & "|#units")="1"
Else
QP_ReferenceAliases.Add Key,Record("uid")
End If
QP_ReferenceAliases(UnitKey)=Record("uid")
End Sub
Sub QPBindSnapshotFullState(ByVal Context, ByVal State)
Dim Line, Fields, Record
For Each Line In Split(State,vbCrLf)
Fields=Split(Line,vbTab)
If UBound(Fields)=7 Then
If Fields(0)="C" Then
Set Record=CreateObject("Scripting.Dictionary")
Record.Add "uid",QPDecode(Fields(1)):Record.Add "refdes",QPDecode(Fields(2))
Record.Add "x",QPDecode(Fields(3)):Record.Add "y",QPDecode(Fields(4))
Record.Add "orientation",QPDecode(Fields(5)):Record.Add "scale",QPDecode(Fields(6))
QPBindSnapshotReference Context,Record
End If
End If
Next
End Sub
Function QPReadInventoryRequest()
Dim Path, Lines, Header, Result, Index, Fields, OldPart, Key
Path = QP_Fso.BuildPath(QP_Root, "inventory-request.tsv")
@@ -943,14 +1084,20 @@ Sub QPCloseBatchProject(ByVal ExpectedPath, ByVal SaveChanges)
If QPCanonicalPath(QPProjectPath()) <> QPCanonicalPath(ExpectedPath) Then _
Err.Raise vbObjectError + 871, "QuickPartReplace.Batch", "active project changed before save and close"
Set Documents = QP_App.SchematicSheetDocuments()
If Documents.Count <> 1 Then Err.Raise vbObjectError + 871, "QuickPartReplace.Batch", "open sheet set changed; no automatic close"
Set Document = Documents.Item(1)
If QP_SessionMode Then
If Documents.Count <> QP_SessionDocuments.Count Then Err.Raise vbObjectError + 871, "QuickPartReplace.Batch", "owned sheet set changed; no automatic close"
Else
If Documents.Count <> 1 Then Err.Raise vbObjectError + 871, "QuickPartReplace.Batch", "open sheet set changed; no automatic close"
End If
QPProgress "save_and_close", 0, 1
If SaveChanges Then
QP_BatchSaveRequested = True
QPLog QP_Root, "INFO", "save_requested job=" & QP_CurrentJobId
End If
Document.Close CBool(SaveChanges), ""
Do While Documents.Count > 0
Set Document = Documents.Item(1)
Document.Close CBool(SaveChanges), ""
Loop
If SaveChanges Then
QP_BatchSaveCommitted = True
QPLog QP_Root, "INFO", "save_returned job=" & QP_CurrentJobId
@@ -968,14 +1115,19 @@ Sub QPCloseBatchProject(ByVal ExpectedPath, ByVal SaveChanges)
End Sub
Sub QPDiscardFailedBatchProject(ByRef ErrorDescription)
Dim ErrorNumber, ErrorSource, Description
Dim ErrorNumber, ErrorSource, Description, HasStagedItems
ErrorDescription = ""
If (Not QP_BatchMode And Not QP_SessionMode) Or Not QP_OwnedProject Or QP_App Is Nothing Then Exit Sub
If QP_NativeMutationPossible Or QP_BatchSaveRequested Or QP_BatchSaveCommitted Then
HasStagedItems = False
If IsObject(QP_SessionItems) Then HasStagedItems = (QP_SessionItems.Count > 0)
If QP_NativeMutationPossible Or QP_BatchSaveRequested Or QP_BatchSaveCommitted Or HasStagedItems Then
ErrorDescription = "write or save outcome requires reconciliation; owned project retained without automatic save or discard"
Exit Sub
End If
' A pending pause must not interrupt the verification and close it requested.
QP_FailureCleanupActive = True
On Error Resume Next
Err.Clear
QPVerifyFailedOriginalSnapshot
If Err.Number = 0 Then QPCloseBatchProject QP_OwnedProjectPath, False
ErrorNumber = Err.Number
@@ -983,6 +1135,7 @@ Sub QPDiscardFailedBatchProject(ByRef ErrorDescription)
Description = Err.Description
Err.Clear
On Error GoTo 0
QP_FailureCleanupActive = False
If ErrorNumber <> 0 Then
ErrorDescription = CStr(ErrorSource) & ": " & CStr(Description)
Else
@@ -1018,6 +1171,10 @@ End Sub
Sub QPFinishBatchItem()
Dim Command, SnapshotPath, Records, Plans, ValueStates, Index, Record, Candidate, TargetPath, FinalState, ValueReportState, SavedSemanticState
If QP_SessionMode Then
QPStageSessionItem
Exit Sub
End If
If QP_BatchState <> "restored_ok" And QP_BatchState <> "no_match" Then _
Err.Raise vbObjectError + 871, "QuickPartReplace.Batch", "worker has no verified terminal result"
FinalState = QP_BatchState
@@ -1100,14 +1257,144 @@ Sub QPFinishBatchItem()
QPProgress "finished", 1, 1
End Sub
Sub QPStageSessionItem()
Dim Command, Records, ValueStates, Item, Index, Record, Candidate, FinalState, Code
If QP_BatchState <> "restored_ok" And QP_BatchState <> "no_match" Then _
Err.Raise vbObjectError + 871, "QuickPartReplace.Batch", "worker has no verified result to stage"
FinalState = QP_BatchState
QP_BatchState = ""
Set Command = QPReadCommand(QP_Fso.BuildPath(QP_Root, "command.tsv"))
QP_BatchState = FinalState
Set Item = CreateObject("Scripting.Dictionary")
Item.Add "command", Command
Item.Add "schematic", CStr(QP_App.ActiveView.GetName(0))
Item.Add "sheet", CStr(QP_App.ActiveView.Block.SheetNum)
Item.Add "final_state", FinalState
If FinalState = "restored_ok" Then
Set Records = QPReadSnapshot(QPJobSnapshotPath(QP_CurrentJobId))
QPRequireCommandSnapshot Command, QPJobSnapshotPath(QP_CurrentJobId)
QPVerifyUntouched Records
Set ValueStates = CreateObject("Scripting.Dictionary")
For Index = 0 To Records.Item("components").Count - 1
Set Record = Records.Item("components").Item(CStr(Index))
Set Candidate = QPFindUniqueByOldLocation(Record)
ValueStates.Add CStr(Index), QPCaptureValueState(Candidate)
QPSetRequiredNc ValueStates.Item(CStr(Index)), Record, Records.Item("attributes"), QPSyncNc(Command)
ValueStates.Item(CStr(Index)).Add "custom_value", QPCustomValue(Command)
Next
Item.Add "snapshot_path", QPJobSnapshotPath(QP_CurrentJobId)
Item.Add "value_states", ValueStates
End If
QP_SessionItems.Add QP_CurrentJobId, Item
QP_SessionSheets.Item(QPSheetKey(Item("schematic"), Item("sheet"))) = QPCurrentSemanticState()
QP_BatchMode = False
QP_NativeMutationPossible = False
QP_SessionExternalChange = False
QP_CheckpointSchematic = Item("schematic")
QP_CheckpointSheet = Item("sheet")
QP_SessionWaiting = True
Code = "session_staged"
If FinalState = "no_match" Then Code = "session_staged_no_match"
QPWriteStatus QP_Root, QP_CurrentJobId, "pending_save", Code, "Verified in memory; awaiting engineering project save and readback."
QPWriteUtf8 QP_Fso.BuildPath(QP_Root, "project-session-result.tsv"), _
QP_SessionToken & vbTab & QP_CurrentJobId & vbTab & "pending_save" & vbTab & Code
End Sub
Sub QPCommitSessionProject()
Dim Job, Item, Command, Records, Plans, ValueStates, Index, Record, Candidate, Key
Dim Documents, Document, Manifest, FinalState, OriginalJob, OriginalSchematic, OriginalSheet, VerifiedSheets, HasChanges
If QP_SessionItems.Count = 0 Then
QPCloseBatchProject QP_SessionProject, False
Exit Sub
End If
OriginalJob = QP_CurrentJobId
OriginalSchematic = QP_TargetSchematic
OriginalSheet = QP_TargetSheet
QP_SessionWaiting = False
QP_SessionCommitActive = True
QP_FailureCleanupActive = True
QP_BatchMode = True
HasChanges = False
Set VerifiedSheets = CreateObject("Scripting.Dictionary")
' Capture the final memory state once per sheet, including later replacements.
For Each Job In QP_SessionItems.Keys
Set Item = QP_SessionItems.Item(Job)
QP_TargetSchematic = Item("schematic")
QP_TargetSheet = Item("sheet")
Key = QPSheetKey(QP_TargetSchematic, QP_TargetSheet)
If Not VerifiedSheets.Exists(Key) Then
QPEnsureProject QP_App, QP_SessionProject
QPRequireExpectedState QP_SessionSheets.Item(Key), QPCurrentSemanticState()
VerifiedSheets.Add Key, True
End If
If Item("final_state") = "restored_ok" Then HasChanges = True
Next
QPCloseBatchProject QP_SessionProject, HasChanges
QP_SessionDocuments.RemoveAll
VerifiedSheets.RemoveAll
QPProgress "verify_saved_project", 0, QP_SessionItems.Count
Manifest = QP_SessionToken & vbTab & QPEncode(QP_SessionProject) & vbTab & "project_verified" & vbCrLf
For Each Job In QP_SessionItems.Keys
Set Item = QP_SessionItems.Item(Job)
Set Command = Item("command")
QP_CurrentJobId = CStr(Job)
QP_TargetSchematic = Item("schematic")
QP_TargetSheet = Item("sheet")
QPEnsureProject QP_App, QP_SessionProject
Key = QPSheetKey(QP_TargetSchematic, QP_TargetSheet)
If Not VerifiedSheets.Exists(Key) Then
QPRequireExpectedState QP_SessionSheets.Item(Key), QPCurrentSemanticState()
VerifiedSheets.Add Key, True
End If
QPWriteSemanticState Command, QP_SessionSheets.Item(Key)
FinalState = "no_match"
If Item("final_state") = "restored_ok" Then
Set Records = QPReadSnapshot(CStr(Item("snapshot_path")))
Set ValueStates = Item("value_states")
Set Plans = CreateObject("Scripting.Dictionary")
QPInvalidatePinIndex
QPInvalidateComponentIndex
For Index = 0 To Records.Item("components").Count - 1
Set Record = Records.Item("components").Item(CStr(Index))
Set Candidate = QPFindUniqueByOldLocation(Record)
If StrComp(CStr(Candidate.RefDes), CStr(Record("refdes")), vbTextCompare) <> 0 Then _
Err.Raise vbObjectError + 871, "QuickPartReplace.Batch", "saved component refdes differs"
Plans.Add CStr(Index), Candidate
Next
QPVerifyRestoredPlans Command, Records, Plans, ValueStates
QPWriteValueReadback Command, Records, Plans, "saved_ok"
FinalState = "saved_ok"
End If
Manifest = Manifest & CStr(Job) & vbTab & FinalState & vbCrLf
Next
QPCloseBatchProject QP_SessionProject, False
' This durable marker appears only after every readback and final close passes.
QPWriteUtf8 QP_Fso.BuildPath(QP_Root, "project-commit-" & QP_SessionToken & ".tsv"), Manifest
For Each Job In QP_SessionItems.Keys
Set Item = QP_SessionItems.Item(Job)
FinalState = "no_match"
If Item("final_state") = "restored_ok" Then FinalState = "saved_ok"
QPWriteStatus QP_Root, CStr(Job), FinalState, "project_committed", "Engineering project saved, reopened, readback verified, and closed."
Next
QP_CurrentJobId = OriginalJob
QP_TargetSchematic = OriginalSchematic
QP_TargetSheet = OriginalSheet
QP_SessionCommitActive = False
QP_FailureCleanupActive = False
QP_BatchMode = False
End Sub
Function QPCurrentSemanticState()
Dim Components, Comp, State
Dim Components, Comp, State, Count
QPInvalidatePinIndex
QPInvalidateComponentIndex
Set Components = QP_App.ActiveView.Query(VDM_COMP, VD_ALL)
State = ""
Count = 0
For Each Comp In Components
State = State & QPUntouchedFullState(Comp)
Count = Count + 1
QPProgress "capture_semantic_state", Count, Components.Count
Next
QPCurrentSemanticState = "S" & vbTab & "full_sheet_v1" & vbCrLf & QPSortSnapshotLines(State)
End Function
@@ -1194,14 +1481,18 @@ Sub QPWriteSemanticState(ByRef Command, ByVal SemanticState)
End Sub
Sub QPVerifyExpectedBaselines(ByVal TargetPath)
Dim ErrorNumber, Description
Dim ErrorNumber, Description, ErrorSource
On Error Resume Next
QPCheckExpectedBaselines TargetPath
ErrorNumber = Err.Number
Description = Err.Description
ErrorSource = Err.Source
Err.Clear
On Error GoTo 0
If ErrorNumber <> 0 Then Err.Raise vbObjectError + 879, "QuickPartReplace.Baseline", Description
If ErrorNumber <> 0 Then
If ErrorSource = "QuickPartReplace.SafeStop" Then Err.Raise ErrorNumber, ErrorSource, Description
Err.Raise vbObjectError + 879, "QuickPartReplace.Baseline", Description
End If
End Sub
Sub QPCheckExpectedBaselines(ByVal TargetPath)
@@ -1408,7 +1699,7 @@ Sub QPCaptureAndSelect(ByRef Command, ByVal SnapshotPath)
Dim PartNumber
Dim Snapshot
Dim Index
Dim Selectors, Selector, Uid, ValueAttr, ValueText, MatchedUids, UntouchedCount, NativeRead, AlreadyTargets
Dim Selectors, Selector, SelectorUid, FoundSelectors, Uid, ValueAttr, ValueText, MatchedUids, UntouchedCount, NativeRead, AlreadyTargets
QP_FallbackIdentityUsed = False
Set QP_FallbackIdentityWarnings = CreateObject("Scripting.Dictionary")
@@ -1419,22 +1710,27 @@ Sub QPCaptureAndSelect(ByRef Command, ByVal SnapshotPath)
QPProgress "query_components", 0, 0
Set Matches = CreateObject("Scripting.Dictionary")
Set AlreadyTargets = CreateObject("Scripting.Dictionary")
Set FoundSelectors = CreateObject("Scripting.Dictionary")
QP_Stage = "query_components"
QPLog QP_Root, "INFO", "stage=query_components"
Set Components = QP_App.ActiveView.Query(VDM_COMP, VD_ALL)
' Capture walks the queried collection directly. Build lookup indexes only
' when restoration needs them, from fresh proxies after native mutation.
Set Selectors = QPCommandTargets(Command)
QPBindSheetReferences Components
For Each Comp In Components
Uid = QPComponentIdentityKey(Comp)
SelectorUid=""
If Not Selectors Is Nothing Then SelectorUid=QPCommandTargetKey(Comp,Selectors)
If Selectors Is Nothing Then
PartNumber = QPReadableAttributeValue(Comp, "Part Number")
If StrComp(PartNumber, CStr(Command("old_part_number")), _
vbTextCompare) = 0 Then Matches.Add CStr(Matches.Count), Comp
ElseIf Selectors.Exists(Uid) Then
ElseIf Len(SelectorUid)>0 Then
FoundSelectors(SelectorUid)=True
QPRequireUniqueAttributeNames Comp
PartNumber = QPRequiredAttributeValue(Comp, "Part Number")
Selector = Selectors.Item(Uid)
Selector = Selectors.Item(SelectorUid)
ValueText = ""
Set ValueAttr = QPFindCurrentAttribute(Comp, "Value")
If Not ValueAttr Is Nothing Then ValueText = QPEffectiveAttributeValue(ValueAttr)
@@ -1452,7 +1748,7 @@ Sub QPCaptureAndSelect(ByRef Command, ByVal SnapshotPath)
QPProgress "find_components", Matches.Count, 0
Next
If Not Selectors Is Nothing Then
If Matches.Count + AlreadyTargets.Count <> Selectors.Count Then _
If FoundSelectors.Count <> Selectors.Count Then _
Err.Raise vbObjectError + 873, "QuickPartReplace.Target", "selected component is missing"
If AlreadyTargets.Count > 0 Then
If Matches.Count > 0 Then Err.Raise vbObjectError + 880, "QuickPartReplace.Target", _
@@ -1545,13 +1841,24 @@ Sub QPCaptureAndSelect(ByRef Command, ByVal SnapshotPath)
End Sub
Sub QPRequireNativeCapturedTarget(ByVal State, ByRef Command, ByVal Uid, ByRef Selectors)
Dim Fields, Selector
Dim Fields, Selector, Key, Found
Fields = Split(QPUntouchedIdentity(State, True), vbTab)
If QPDecode(Fields(1)) <> Uid Or StrComp(QPDecode(Fields(3)), _
CStr(Command("old_part_number")), vbTextCompare) <> 0 Then _
Err.Raise vbObjectError + 873, "QuickPartReplace.Target", "selected component changed during native capture"
If Not Selectors Is Nothing Then
Selector = Selectors(Uid)
Found=False
For Each Key In Selectors.Keys
Selector=Selectors(Key)
If StrComp(QPDecode(Fields(2)),CStr(Selector(0)),vbTextCompare)=0 Then
If Found Then Err.Raise vbObjectError+873,"QuickPartReplace.Target","ambiguous physical selector"
Found=True
If QPDecode(Fields(4))<>CStr(Selector(1)) Then Err.Raise vbObjectError+873, _
"QuickPartReplace.Target","selected unit Value changed during capture"
End If
Next
If Not Found Then Err.Raise vbObjectError+873,"QuickPartReplace.Target","foreign selected unit reference"
Exit Sub
If QPDecode(Fields(2)) <> CStr(Selector(0)) Or QPDecode(Fields(4)) <> CStr(Selector(1)) Then _
Err.Raise vbObjectError + 873, "QuickPartReplace.Target", "selected component changed during native capture"
End If
@@ -1573,6 +1880,19 @@ Function QPUseNativeSnapshot()
QPUseNativeSnapshot = True
End Function
Function QPCommandTargetKey(ByRef Comp, ByRef Selectors)
Dim Key, Value, Found
Found=""
For Each Key In Selectors.Keys
Value=Selectors(Key)
If StrComp(Trim(CStr(Comp.Refdes)),Trim(CStr(Value(0))),vbTextCompare)=0 Then
If Len(Found)>0 Then Err.Raise vbObjectError+873,"QuickPartReplace.Target","duplicate physical target reference"
Found=CStr(Key)
End If
Next
QPCommandTargetKey=Found
End Function
Function QPReadableAttributeValue(ByRef Comp, ByVal Name)
Dim Attr, ErrorNumber, ErrorSource, ErrorDescription
On Error Resume Next
@@ -1743,7 +2063,7 @@ Sub QPVerifyUntouched(ByRef Records)
End Sub
Function QPCommandTargets(ByRef Command)
Dim Scope, Text, Entries, Entry, Fields, Result, Index
Dim Scope, Text, Entries, Entry, Fields, Result, Index, AliasKey
Set QPCommandTargets = Nothing
If Not Command.Exists("target_scope") Then Exit Function
Scope = CStr(Command("target_scope"))
@@ -1767,6 +2087,11 @@ Function QPCommandTargets(ByRef Command)
If Result.Exists(CStr(Fields(0))) Then Err.Raise vbObjectError + 873, _
"QuickPartReplace.Target", "duplicate target UID"
Result.Add CStr(Fields(0)), Array(CStr(Fields(1)), CStr(Fields(2)))
If Command.Exists("schematic_name") And Command.Exists("sheet_number") Then
If Not IsObject(QP_ReferenceAliases) Then Set QP_ReferenceAliases=CreateObject("Scripting.Dictionary")
AliasKey=LCase(Command("schematic_name") & ":" & Command("sheet_number") & "|" & Trim(Fields(1)))
QP_ReferenceAliases(AliasKey)=CStr(Fields(0))
End If
Next
Set QPCommandTargets = Result
End Function
@@ -3363,7 +3688,7 @@ Function QPReadProjectRuleStates(ByRef Lines, ByRef Keys, ByVal StatusPath)
End Function
Function QPIsNextRuleState(ByVal State)
QPIsNextRuleState = (Len(CStr(State)) = 0)
QPIsNextRuleState = (Len(CStr(State)) = 0 Or CStr(State) = "pending")
End Function
Function QPJobSnapshotPath(ByVal JobId)
@@ -3467,6 +3792,7 @@ Function QPReadSnapshot(ByVal Path)
Record.Add "orientation", QPDecode(Fields(5))
Record.Add "scale", QPDecode(Fields(6))
Record.Add "connections", QPDecode(Fields(7))
If Result.Exists("meta") Then QPBindSnapshotReference Meta("sheet_name"),Record
Components.Add CStr(Components.Count), Record
Case "A"
If UBound(Fields) <> 11 Then Err.Raise vbObjectError + 814, _
@@ -3490,6 +3816,11 @@ Function QPReadSnapshot(ByVal Path)
If Untouched.Exists(QPDecode(Fields(1))) Then Err.Raise vbObjectError + 874, _
"QuickPartReplace.Target", "duplicate unselected component UID"
Untouched.Add QPDecode(Fields(1)), Replace(Lines(Index), vbCr, "")
If Result.Exists("meta") And Len(Trim(QPDecode(Fields(2))))>0 Then
If Not IsObject(QP_ReferenceAliases) Then Set QP_ReferenceAliases=CreateObject("Scripting.Dictionary")
If Not QP_ReferenceAliases.Exists(LCase(Meta("sheet_name") & "|" & Trim(QPDecode(Fields(2))))) Then _
QP_ReferenceAliases.Add LCase(Meta("sheet_name") & "|" & Trim(QPDecode(Fields(2)))),QPDecode(Fields(1))
End If
Case "X"
If UBound(Fields) <> 1 Or Fields(1) <> "untouched_full_v1" Then _
Err.Raise vbObjectError + 874, "QuickPartReplace.Target", "invalid unselected-state protocol"
@@ -3498,6 +3829,7 @@ Function QPReadSnapshot(ByVal Path)
If UBound(Fields) <> 2 Or Len(QPDecode(Fields(1))) = 0 Or Len(QPDecode(Fields(2))) = 0 Then _
Err.Raise vbObjectError + 874, "QuickPartReplace.Target", "invalid unselected full-state snapshot"
UntouchedFull.Add QPDecode(Fields(1)), QPDecode(Fields(2))
If Result.Exists("meta") Then QPBindSnapshotFullState Meta("sheet_name"),QPDecode(Fields(2))
End Select
End If
Next
+2 -1
View File
@@ -286,7 +286,8 @@ def _execute_incident_retry(service, root, task_id, run_id, *, expected_item, st
raise TaskCenterError('run_scope', '原队列检查点身份或范围已变化;保留本次结果。')
checkpoint['entries'][task_id].update(state=state, message=message)
if path.parent.name == 'workspace-queues':
checkpoint['entries'][task_id].update(released=not bool(service.pending_batch_resources()), problems_done=True)
from ..trial_task_center.batch_resources import resource_blockers
checkpoint['entries'][task_id].update(released=not bool(resource_blockers(service.pending_batch_resources())), problems_done=True)
if reports:
checkpoint['entries'][task_id]['netlist_verification'] = reports[task_id]
elif reports:
+2 -1
View File
@@ -316,7 +316,8 @@ class IncidentRunner:
remaining = tuple(self.service.pending_batch_resources())
cleanup['remaining_resources'] = _bounded_details(remaining)
cleanup['remaining_resources_observed'] = True
released = not remaining
from ..trial_task_center.batch_resources import resource_blockers
released = not resource_blockers(remaining)
except Exception as error:
cleanup['verification_error'] = scrub({'type': type(error).__name__, 'detail': str(error)[:1000]})
cleanup['remaining_resources'], released = None, None
+2 -1
View File
@@ -402,7 +402,8 @@ class RunJournal:
blockers.append("preflight_blocked")
except Exception:
blockers.append("preflight_unavailable")
if service.pending_batch_resources():
from ..trial_task_center.batch_resources import resource_blockers
if resource_blockers(service.pending_batch_resources()):
blockers.append("resources_pending")
blockers = list(dict.fromkeys(blockers))
value = {**row, "queue_available": queue is not None, "tasks": tasks, "counters": counters,
+2 -1
View File
@@ -246,7 +246,8 @@ class ScheduleStore:
with self.execution_lease(schedule_id):
assert_task_queue_idle(self.data_root)
with _queue_lease(self.data_root):
if service.pending_batch_resources():
from ..trial_task_center.batch_resources import resource_blockers
if resource_blockers(service.pending_batch_resources()):
raise TaskCenterError('schedule_busy', '后台资源尚未释放,请先在任务中心核验和清理。')
with self._connect() as db:
db.execute('BEGIN IMMEDIATE')
+2 -1
View File
@@ -50,7 +50,8 @@ def preflight(service, task_ids, *, allow_completed=False):
from .impact import _scope
task_ids = _scope(task_ids)
tasks, paths, hashes = [], set(), {}
pending = bool(service.pending_batch_resources())
from ..trial_task_center.batch_resources import resource_blockers
pending = bool(resource_blockers(service.pending_batch_resources()))
for task_id in task_ids:
context = _context(service, task_id)
state = execution_state(context)
+10 -3
View File
@@ -121,6 +121,9 @@ def execute_batch(
**({'runtime': runtime} if runtime is not None else {})) as project_session:
while not progress.queue_complete:
if stop_event.is_set():
if project_session.enabled and project_session.worker is not None:
progress = preparer.complete_project_session(task_id, project_session, session, projects)
progress = service._synchronize_quick_run_progress(task_id, progress=progress)
backups.commit_prefetched(progress)
return BatchRunResult(
progress, True, last_result, tuple(item_messages), tuple(backups.unbacked_project_paths)
@@ -133,6 +136,9 @@ def execute_batch(
backups.prepare(current, progress)
service._batch_feedback = (task_id, progress, f'正在处理 {current.name}')
if stop_event.is_set():
if project_session.enabled and project_session.worker is not None:
progress = preparer.complete_project_session(task_id, project_session, session, projects)
progress = service._synchronize_quick_run_progress(task_id, progress=progress)
backups.commit_prefetched(progress)
return BatchRunResult(
progress, True, last_result, tuple(item_messages), tuple(backups.unbacked_project_paths)
@@ -197,7 +203,7 @@ def execute_batch(
candidate = last_result.progress
project_completed = candidate.completed_project_paths != progress.completed_project_paths
synchronize = (
progress.completed_items == 0
(progress.completed_items == 0 and last_result.state != 'pending_save')
or project_completed
or candidate.queue_complete
or last_result.state in {BATCH_FAILED_STATE, BATCH_SAVED_UNVERIFIED_STATE}
@@ -208,7 +214,8 @@ def execute_batch(
persistence['completed_units'] = 1
else:
updated = candidate
if updated.completed_items <= progress.completed_items or updated.total_items != progress.total_items:
if ((updated.completed_items <= progress.completed_items and last_result.state != 'pending_save')
or updated.total_items != progress.total_items):
_write_uncertain_flag(preparer.data_root / 'quick-run/uncertain.flag', 'batch_progress_mismatch')
raise TaskCenterError('quick_run_queue_progress', '执行器返回后队列进度异常,已停止,避免重复执行。')
# Keep every verified step: one project can contain many sheets and rules.
@@ -266,7 +273,7 @@ def execute_batch(
feedback = '未命中已跳过,继续下一项' if last_result.state == 'no_match' else (
'失败已记录,继续下一项'
if last_result.state == BATCH_FAILED_STATE
else '已保存并校验,继续下一项'
else '替换已暂存,等待工程统一保存回读' if last_result.state == 'pending_save' else '已保存并校验,继续下一项'
)
service._batch_feedback = (task_id, progress, feedback)
return BatchRunResult(
@@ -31,7 +31,7 @@ def write_release_event(root, path, record, phase, *, context=None, reason='batc
'generation': record.get('generation'), 'desktop': record.get('desktop'),
'cad_pid': record.get('cad_pid'), 'cad_guid': record.get('cad_guid'),
'reason': reason, 'result': 'releasing' if phase == 'started' else (
'released' if phase == 'succeeded' else 'blocked'),
'released' if phase == 'succeeded' else 'isolated' if phase == 'isolated' else 'blocked'),
'elapsed_seconds': elapsed, 'record_path': str(path),
'desktop_handle_closed': cleanup.get('desktop_handle_closed'),
'desktop_disappeared': disappeared, 'cad_exit_code': cad.get('cad_exit_code'),
@@ -39,7 +39,8 @@ def write_release_event(root, path, record, phase, *, context=None, reason='batc
'cad_shutdown': cad, 'shutdown_warning': record.get('shutdown_warning'),
'retries': record.get('retries', {}), 'cleanup': cleanup, 'error': str(error)}
title = {'started': '开始释放后台资源', 'succeeded': '后台资源释放成功',
'failed': '后台资源释放失败,资源释放待处理'}[phase]
'failed': '后台资源释放失败,资源释放待处理',
'isolated': '后台写入进程已退出,残留桌面已隔离(未确认释放)'}[phase]
purpose = {'batch_end': '批次结束', 'rebuild': '安全重建', 'manual_retry': '重新清理',
'restart_recheck': '重启后重新检查'}.get(reason, reason)
cad_text = f"CAD PID={details['cad_pid']}" if details['cad_pid'] else (
@@ -56,7 +57,7 @@ def write_release_event(root, path, record, phase, *, context=None, reason='batc
if error:
message += ' · 原因:' + str(error)
message = message.replace('\r', '\\r').replace('\n', '\\n')
level = 'ERROR' if phase == 'failed' else 'INFO'
level = 'ERROR' if phase == 'failed' else 'WARNING' if phase == 'isolated' else 'INFO'
timestamp = datetime.now(timezone.utc).isoformat(timespec='milliseconds')
errors = []
try:
@@ -51,7 +51,7 @@ def _valid_record(record, path):
and type(record.get('generation')) is int and record['generation'] >= 1
and isinstance(record.get('desktop'), str)
and re.fullmatch('dx-[0-9a-f]{32}', record['desktop']) is not None
and record.get('state') in {'starting', 'ready', 'recycling', 'closing', 'blocked', 'released'}
and record.get('state') in {'starting', 'ready', 'recycling', 'closing', 'blocked', 'released', 'isolated'}
and isinstance(processes, list) and all(isinstance(item, dict)
and type(item.get('pid')) is int and item['pid'] > 0
and 'creation_time' in item and (item['creation_time'] is None
@@ -66,14 +66,17 @@ def _valid_record(record, path):
def pending_resources(root):
pending = []
for path in sorted(internal_path(canonical_path(root) / 'batch-resources').glob('*/session.json')):
directory = internal_path(canonical_path(root) / 'batch-resources')
paths = (*directory.glob('*/session.json'), *directory.glob('*/isolated-generation-*.json'))
for path in sorted(paths):
path = canonical_path(path)
try:
record = json.loads(internal_path(path).read_text(encoding='utf-8'))
if not _valid_record(record, path):
raise ValueError('invalid resource ownership record')
if record.get('state') != 'released' or not _released_evidence(record):
pending.append(dict(record, record_path=str(path)))
pending.append(dict(record, record_path=str(path),
blocking=not (record.get('state') == 'isolated' and _isolation_evidence(record))))
except (OSError, ValueError, TypeError) as error:
pending.append({'state': 'blocked', 'record_path': str(path), 'error': str(error)})
import sys
@@ -109,6 +112,57 @@ def pending_resources(root):
return tuple(pending)
def resource_blockers(records):
"""Only a validated isolated desktop warning may leave the resource gate open."""
return tuple(item for item in records
if not (item.get('state') == 'isolated' and item.get('blocking') is False))
def blocking_resources(root):
return resource_blockers(pending_resources(root))
def writer_identities(record):
identities = []
for item in [*record.get('writer_identities', ()), *record.get('processes', ()),
*([record['cad_process']] if record.get('cad_process') else [])]:
if item not in identities:
identities.append(item)
if record.get('cad_pid') is not None and not any(
item.get('pid') == record['cad_pid'] for item in identities):
raise ValueError('CAD process identity is missing')
if any(type(item.get('pid')) is not int or item['pid'] <= 0
or not isinstance(item.get('creation_time'), str) or not item['creation_time']
for item in identities):
raise ValueError('Exact writer process identity is missing')
return identities
def isolation_proof(record):
"""Certify handle closure and exact writer exit, never a desktop disappearance."""
cleanup = record.get('cleanup') or {}
if (cleanup.get('desktop_handle_closed') is not True
or cleanup.get('owned_cleanup_complete') is not True):
raise ValueError('Owned resource cleanup is not proven')
identities = writer_identities(record)
if not all(_process_gone(item) for item in identities):
raise ValueError('Owned writer process is still active')
return {'session_id': record['session_id'], 'generation': record['generation'],
'desktop': record['desktop'], 'desktop_handle_closed': True, 'desktop_released': False,
'processes_exited': True, 'processes': identities, 'observed_at': time.time()}
def _isolation_evidence(record):
try:
proof = record.get('isolation_proof', {})
current = isolation_proof(record)
return (isinstance(proof, dict) and all(proof.get(key) == current[key] for key in
('session_id', 'generation', 'desktop', 'desktop_handle_closed', 'desktop_released',
'processes_exited', 'processes')))
except Exception:
return False
def _released_evidence(record):
"""A state label alone must never reopen the physical resource gate."""
proof = record.get('release_proof', {})
@@ -126,7 +180,7 @@ def _released_evidence(record):
def guard_pending_resources(root):
if pending_resources(root):
if blocking_resources(root):
raise BatchResourceCleanupError('上一批资源尚未确认释放;请先重试释放资源,再开始新批次。')
@@ -216,7 +270,7 @@ def cleanup_pending_resources(root, *, logging_context=None, retry_failed=False)
if not record.get('session_id'):
continue # Corrupt evidence cannot authorize resource actions.
log('started')
identities = list(record.get('processes', ()))
identities = writer_identities(record)
cad_identity = record.get('cad_process')
if record.get('cad_pid') is not None:
if (not isinstance(cad_identity, dict) or cad_identity.get('pid') != record['cad_pid']
@@ -230,11 +284,24 @@ def cleanup_pending_resources(root, *, logging_context=None, retry_failed=False)
continue
desktop = record.get('desktop')
if desktop:
_Win32Api().confirm_desktop_released(desktop)
try:
_Win32Api().confirm_desktop_released(desktop)
except Exception as error:
# A residual desktop name is only a warning after owned
# handle closure and independently revalidated writer exit.
proof = isolation_proof(record)
record.pop('record_path', None)
record.update(state='isolated', blocking=False, isolation_proof=proof,
isolation_warning=str(error))
write_record(path, record)
log('isolated', str(error))
continue
# After restart only confirm disappearance. Never attach to COM,
# close a project, or terminate a PID whose handles we no longer own.
record.pop('record_path', None)
record = dict(record, state='released', reconciled=True, released_at=time.time(),
record = dict(record, state='released', blocking=False, reconciled=True, released_at=time.time(),
release_results={'mentor_released': True, 'desktop_released': True},
release_state={'mentor': 'released', 'desktop': 'released'}, wait_reason='',
release_proof={'generation': record['generation'], 'desktop_released': True,
'processes_exited': True, 'processes': identities, 'method': 'restart_observation'})
write_record(path, record)
@@ -11,7 +11,8 @@ from threading import Lock
from uuid import UUID, uuid4
from .batch_resources import (BatchResourceCleanupError, LIVE_RESOURCES, cleanup_pending_resources,
guard_pending_resources, pending_resources, process_identity, project_checkpoint_released, write_record)
guard_pending_resources, pending_resources, process_identity, project_checkpoint_released, write_record,
isolation_proof)
from .errors import TaskCenterError
from .path_access import canonical_path, internal_path, legacy_path
from .isolated_desktop import PrivateDesktop, ProcessNotStartedError
@@ -202,12 +203,18 @@ class BatchCadRuntime:
def _ensure_desktop(self, abort_event=None):
if self.record['state'] == 'blocked':
raise BatchResourceCleanupError('当前批次资源已阻断,不能启动新的工作进程。')
if self.record['state'] == 'isolated':
if self.desktop is not None:
raise BatchResourceCleanupError('隔离桌面已停止使用,请为后续工程创建新的运行会话。')
write_record(self.path.parent / f'isolated-generation-{self.generation}.json', self.record)
if self.desktop is None:
check_cancelled(self.stop_event if abort_event is None else abort_event)
self.generation += 1
name = 'dx-' + uuid4().hex
self._save(state='starting', generation=self.generation, desktop=name,
processes=[], cad_pid=None, cad_guid=None, checkpoints={}, native_operations={},
processes=[], writer_identities=[], cad_pid=None, cad_guid=None, checkpoints={}, native_operations={},
blocking=True, isolation_proof=None,
stopping_reason='', wait_reason='', primary_error=None,
cad_process=None, cad_token=None, cad_shutdown=None, cad_start_requested=False, cleanup=None,
release_state={'mentor': 'not_started', 'desktop': 'pending'})
LIVE_RESOURCES[str(self.path)] = self
@@ -226,6 +233,8 @@ class BatchCadRuntime:
identity = process_identity(process)
if identity not in self.record['processes']:
self.record['processes'].append(identity)
if identity not in self.record.setdefault('writer_identities', []):
self.record['writer_identities'].append(identity)
self._save()
def _release_process(self, process, *, desktop=None, generation=None):
@@ -245,6 +254,9 @@ class BatchCadRuntime:
cad_process = getattr(self.cad, 'cad', None)
if cad_process is not None and cad_process not in processes:
processes.append(cad_process)
for process in getattr(self.cad, '_retained_processes', ()):
if process not in processes:
processes.append(process)
if not processes:
cleanup = getattr(self.desktop, 'cleanup_record', None) or {}
# Closing our handle follows owned-process cleanup. An external
@@ -403,10 +415,47 @@ class BatchCadRuntime:
key = str(root).casefold()
self.record['checkpoints'][key] = attempt_id
self._save(state='ready')
write_record(root / 'project-checkpoint.json', {'version': 1, 'resource_record': str(self.path),
checkpoint = {'version': 1, 'resource_record': str(self.path),
'session_id': self.session_id, 'generation': self.generation, 'attempt_id': attempt_id,
'evidence_root': str(root), 'project_path': str(project), 'project_closed': True,
'worker_closed': True, 'cad_pid': self.record['cad_pid'], 'cad_guid': self.record['cad_guid']})
'worker_closed': True, 'cad_pid': self.record['cad_pid'], 'cad_guid': self.record['cad_guid']}
request_path = internal_path(root / 'project-session.request')
if request_path.is_file():
request = request_path.read_text(encoding='utf-8-sig').strip().split('\t')
if (len(request) == 3 and len(request[0]) == 32
and all(character in '0123456789abcdef' for character in request[0])
and canonical_path(request[1]) == project):
marker = internal_path(root / ('project-commit-' + request[0] + '.tsv'))
if marker.is_file():
from .execution_proof import capture_project_commit, bind_project_commit_source
from .quick_run import _decode_vbs_field
rows = [line.split('\t') for line in marker.read_text(encoding='utf-8-sig').splitlines() if line]
if (not rows or len(rows[0]) != 3 or rows[0][0] != request[0] or rows[0][2] != 'project_verified'
or canonical_path(_decode_vbs_field(rows[0][1])) != project
or any(len(row) != 2 for row in rows[1:])):
raise WorkerMonitorError('native project commit marker is not bound to the closed project')
jobs = [row[0] for row in rows[1:]]
expected_jobs = [record['job_id'] for _, record in self._current_attempts]
expected_jobs = expected_jobs[expected_jobs.index(self.record['active_job']):]
if not jobs or len(set(jobs)) != len(jobs) or jobs != expected_jobs[:len(jobs)]:
raise WorkerMonitorError('native project commit marker does not cover an authorized prefix')
proof = capture_project_commit(self.root, root, {'session_token': request[0],
'project_path': str(project).casefold(), 'jobs_json': json.dumps(jobs)})
proof = bind_project_commit_source(proof, self.root, root, project)
proof['marker_sha256'] = hashlib.sha256(marker.read_bytes()).hexdigest()
proof['files'].append({'path': str(canonical_path(marker)), 'sha256': proof['marker_sha256']})
readback_names = []
for job, outcome in proof['outcomes'].items():
readback_names.append('post-state-' + job + '.tsv')
if outcome == 'saved_ok':
readback_names.extend(('snapshot-' + job + '.tsv', 'snapshot-' + job + '.tsv.native.json',
'value-readback-' + job + '.tsv'))
proof['readback_files'] = {name: hashlib.sha256(internal_path(root / name).read_bytes()).hexdigest()
for name in readback_names}
proof['native_attempt'] = {'session_id': self.session_id, 'generation': self.generation,
'attempt_id': attempt_id}
checkpoint['project_commit_source'] = proof
write_record(root / 'project-checkpoint.json', checkpoint)
self._active_attempts = []
def _certify_unlaunched_worker(self, root, project, command, attempt_id, expected, before, inputs, *, failure=None,
@@ -514,6 +563,11 @@ class BatchCadRuntime:
cooperative_abort=cooperative_abort, process_factory=start_worker,
**({'owner_loss_path': self.path.parent / 'owner-loss.json'} if self.read_only else {}),
**({'force_control': self.stop_event} if hasattr(self.stop_event, 'force_event') else {}))
if result.returncode == 75 and not self.read_only:
result.prewrite_pause = self._certify_prewrite_pause(
root, project, attempt_id, expected, before, inputs,
getattr(result, 'prewrite_pause', {}), abort_event)
return result
if result.returncode == 74 and not self.read_only:
from .batch_attempt import can_retry_attempt
pause = getattr(result, 'startup_pause', {})
@@ -643,6 +697,87 @@ class BatchCadRuntime:
self.close(rebuilding=True) # Prove disappearance before creating another.
self.desktop = self.cad = None
def _certify_prewrite_pause(self, root, project, attempt_id, expected, before, inputs, pause, abort_event):
from .batch_attempt import can_retry_attempt, read_attempt
from .discovery_cache import project_fingerprint
from .project_session_health import _source_parameters
from .quick_run import _decode_vbs_field
try:
request_path = internal_path(root / 'project-session.request')
request_bytes = request_path.read_bytes()
request = request_bytes.decode('utf-8-sig').strip().split('\t')
original = inputs['project-session.request'].decode('utf-8-sig').strip().split('\t')
if (len(request) != 3 or len(original) != 3 or request[:2] != original[:2]
or pause.get('session_token') != request[0] or len(request[0]) != 32
or pause.get('job_id') != request[2] or canonical_path(request[1]) != project
or canonical_path(pause.get('project_path', '')) != project
or pause.get('worker_tree_exited') is not True
or type(pause.get('sequence')) is not int or pause['sequence'] <= 0
or not ((abort_event is not None and abort_event.is_set())
or (self.stop_event is not None and self.stop_event.is_set()))
or getattr(getattr(self.stop_event, 'force_event', None), 'is_set', lambda: False)()
or internal_path(root / 'command.tsv').read_bytes() != inputs['command.tsv']):
raise ValueError('prewrite pause ownership is not bound')
selected = [(path, record) for path, record in expected if record['job_id'] == request[2]]
if len(selected) != 1 or not can_retry_attempt(*selected[0]):
raise ValueError('current paused job may have mutated')
jobs = [record['job_id'] for _, record in expected]
prefix_jobs = set(jobs[:jobs.index(request[2])])
mutated = set()
for path, record in expected:
actual = read_attempt(path)
if any(actual.get(key) != value for key, value in record.items() if key != 'state'):
raise ValueError('batch attempt identity changed')
if actual['state'] == 'mutation_possible':
mutated.add(record['job_id'])
marker_path = root / ('project-pause-' + request[0] + '.tsv')
marker = internal_path(marker_path).read_text(encoding='utf-8-sig').strip().split('\t')
if (len(marker) != 4 or marker[0] != request[0] or marker[2:] != [request[2], 'prewrite_closed']
or canonical_path(_decode_vbs_field(marker[1])) != project):
raise ValueError('prewrite project closure marker is missing or stale')
manifest_path = root / ('project-commit-' + request[0] + '.tsv')
committed = {}
files = [marker_path, *(path for path, _ in expected)]
if internal_path(manifest_path).exists():
rows = [line.split('\t') for line in internal_path(manifest_path).read_text(encoding='utf-8-sig').splitlines()]
if (not rows or len(rows[0]) != 3 or rows[0][0] != request[0]
or canonical_path(_decode_vbs_field(rows[0][1])) != project or rows[0][2] != 'project_verified'):
raise ValueError('committed prefix manifest is not bound')
for row in rows[1:]:
if (len(row) != 2 or row[0] not in prefix_jobs or row[0] in committed
or row[1] not in {'saved_ok', 'no_match'}):
raise ValueError('committed prefix manifest contains an unrelated result')
committed[row[0]] = row[1]
files.append(manifest_path)
after = project_fingerprint(project, self.root)
if (after is None or not mutated <= committed.keys()
or not committed and (before is None or before != after)):
raise ValueError('source changes lack a verified committed prefix')
self._checkpoint(root, project, attempt_id)
files.append(root / 'project-checkpoint.json')
current = internal_path(root / 'current-command.tsv')
status = internal_path(root / 'status.tsv')
proof = dict(pause, phase='before_native_dispatch', task_id=selected[0][1]['task_id'],
session_id=self.session_id, generation=self.generation, attempt_id=attempt_id,
resource_record=str(self.path), project_closed=True, native_dispatch_not_started=True,
source_sha256=hashlib.sha256(repr(after).encode()).hexdigest(), committed_prefix=committed,
pause_marker_path=str(marker_path),
pause_marker_sha256=hashlib.sha256(internal_path(marker_path).read_bytes()).hexdigest(),
commit_manifest_path=str(manifest_path) if committed else None,
commit_manifest_sha256=hashlib.sha256(internal_path(manifest_path).read_bytes()).hexdigest() if committed else None,
request_sha256=hashlib.sha256(request_bytes).hexdigest(),
status_sha256=hashlib.sha256(status.read_bytes()).hexdigest() if status.exists() else None,
current_command_sha256=hashlib.sha256(current.read_bytes()).hexdigest() if current.exists() else None,
batch_attempts=[{'path': str(path), 'expected': record} for path, record in selected])
proof.update(_source_parameters(project, self.root))
proof_path = root / ('prewrite-pause-' + attempt_id + '.json')
write_record(proof_path, proof)
proof['files'] = [{'path': str(path), 'sha256': hashlib.sha256(internal_path(path).read_bytes()).hexdigest()}
for path in [proof_path, *files]]
return proof
except (OSError, ValueError, TypeError, KeyError, AttributeError) as error:
raise WorkerMonitorError('prewrite pause lacks bound current-job and committed-prefix proof: ' + str(error)) from error
def _retain_uncertain(self, root, project, expected, error):
if self.read_only:
return
@@ -747,7 +882,7 @@ class BatchCadRuntime:
def _close_once(self, *, rebuilding=False, reason=None):
if self.desktop is None and not self.generation:
return # A cache-only batch never acquired resources.
if self.record['state'] == 'released':
if self.record['state'] in {'released', 'isolated'}:
return
reason = reason or ('rebuild' if rebuilding else 'batch_end')
if self._close_failure is not None:
@@ -807,30 +942,41 @@ class BatchCadRuntime:
raise
self._close_failure = None
self._close_failure_rebuild = False
self._release_log('succeeded', reason=reason, elapsed=time.monotonic() - started)
self._release_log('isolated' if self.record['state'] == 'isolated' else 'succeeded',
reason=reason, elapsed=time.monotonic() - started)
def _close_resources(self, *, rebuilding=False):
from .operation_diagnostics import record_operation
operation_root = self._active_evidence or self.path.parent
started = time.monotonic()
errors = []
shutdown_errors = []
retained_cad = False
phases = {'mentor': 'closing' if self.cad is not None else 'released', 'desktop': 'waiting_for_mentor'}
try:
self._save(state='closing', release_state=phases)
self._save(state='closing', release_state=phases,
wait_reason='mentor_exit' if self.cad is not None else 'desktop_release',
stopping_reason=getattr(self.stop_event, 'stopping_reason', ''),
primary_error=self.record.get('primary_error') or getattr(self.stop_event, 'primary_error', ''))
except Exception as error:
errors.append('resource journal: ' + str(error))
if self.cad is not None:
cad = self.cad
try:
failure = getattr(self, '_cad_close_error', None)
record_operation(operation_root, 'mentor_exit', 'enter')
cad.__exit__(type(failure) if failure is not None else None, failure, None)
record_operation(operation_root, 'mentor_exit', 'return')
self.cad = None
except Exception as error:
record_operation(operation_root, 'mentor_exit', 'error', error=str(error))
if rebuilding:
self.record['shutdown_warning'] = str(error)
# A rebuild must not proceed while the previous CAD owner has
# an unconfirmed shutdown. Keep the warning for diagnostics,
# but retain the failure in the resource gate as well.
errors.append(str(error))
shutdown_errors.append(str(error))
from .read_recovery import process_tree_exited
process = getattr(cad, 'cad', None)
processes = [*getattr(cad, '_retained_processes', ()),
@@ -853,9 +999,13 @@ class BatchCadRuntime:
phases['mentor'] = 'failed' if retained_cad else 'released'
if self.desktop is not None and not retained_cad:
phases['desktop'] = 'closing'
self.record['wait_reason'] = 'desktop_release'
try:
record_operation(operation_root, 'desktop_release', 'enter')
self.desktop.close()
record_operation(operation_root, 'desktop_release', 'return')
except Exception as error:
record_operation(operation_root, 'desktop_release', 'error', error=str(error))
errors.append(str(error))
self._timing('batch_release', started)
self.record['cleanup'] = getattr(self.desktop, 'cleanup_record', None)
@@ -868,7 +1018,27 @@ class BatchCadRuntime:
'released' if self.record['release_results']['desktop_released'] else 'failed')
self.record['release_state'] = phases
self.record['cleanup_mode'] = getattr(self.stop_event, 'cleanup_mode', None)
if not retained_cad and not self.record['release_results']['desktop_released']:
try:
proof = isolation_proof(self.record)
if not self._other_writers_exited():
raise ValueError('Retained writer ownership is still active')
except Exception:
pass
else:
phases['desktop'] = 'isolated'
self._save(state='isolated', blocking=False, isolation_proof=proof,
isolation_warning='; '.join(errors), release_state=phases, wait_reason='')
LIVE_RESOURCES.pop(str(self.path), None)
if shutdown_errors:
raise BatchResourceCleanupError('; '.join(shutdown_errors), completed_result=self.result)
return
if errors:
if (errors == shutdown_errors and not retained_cad and self.owned_processes_exited()
and self._other_writers_exited() and self.record['release_results']['desktop_released']):
self.record['shutdown_warning'] = '; '.join(shutdown_errors)
self._record_release()
raise BatchResourceCleanupError('; '.join(errors), completed_result=self.result)
self._save(state='blocked', error='; '.join(errors))
raise BatchResourceCleanupError('; '.join(errors), completed_result=self.result)
self._record_release()
@@ -879,5 +1049,57 @@ class BatchCadRuntime:
'cad_pid': self.record['cad_pid'], 'cad_guid': self.record['cad_guid'],
'cleanup': self.record['cleanup'], 'released_at': released_at}
releases = [*self.record.get('generation_releases', ()), release]
self._save(state='released', released_at=released_at, processes=[], generation_releases=releases)
self._save(state='released', blocking=False, released_at=released_at, processes=[], generation_releases=releases,
wait_reason='')
LIVE_RESOURCES.pop(str(self.path), None)
def _other_writers_exited(self):
from .isolated_cad import RETAINED_CAD_SESSIONS
from .read_recovery import process_tree_exited
from .worker_monitor import RETAINED_WORKERS, _worker_tree_exited
return (all(all(process_tree_exited(process) for process in session._owned_processes())
for session in tuple(RETAINED_CAD_SESSIONS.values())
if session.root.is_relative_to(self.root))
and all(_worker_tree_exited(entry['process'], entry['owned'])
for entry in tuple(RETAINED_WORKERS.values()) if entry['root'].is_relative_to(self.root)))
def release_failed_project(self, error, evidence_root):
"""Release proven stopped writers; the failed project's business evidence stays untouched."""
from .stop_control import protect_interruption
protect_interruption(evidence_root, 'project_failure: ' + str(error))
request = getattr(self.stop_event, 'request_failure_cleanup', None)
if request is not None:
request(evidence_root, error)
if self.cad is not None:
self.cad.stop_control = self.stop_event
self._cad_close_error = error
cleanup_warning = None
try:
self.close(reason='project_failure')
except BatchResourceCleanupError as cleanup:
from .batch_resources import blocking_resources
if (self.record['state'] not in {'released', 'isolated'} or blocking_resources(self.root)
or not self.owned_processes_exited() or not self._other_writers_exited()):
raise
cleanup_warning = str(cleanup)
error.shutdown_warning = cleanup_warning
self.record.setdefault('generation_shutdown_warnings', []).append({
'generation': self.generation, 'error': cleanup_warning, 'cad_shutdown': self.record.get('cad_shutdown')})
self._save(shutdown_warning=cleanup_warning)
if ((self.generation and not self.owned_processes_exited()) or not self._other_writers_exited()):
return False
from .batch_resources import blocking_resources
if blocking_resources(self.root):
return False
write_record(canonical_path(evidence_root) / 'project-resource-release.json', {
'version': 1, 'session_id': self.session_id, 'generation': self.generation,
'resource_record': str(self.path), 'writers_exited': True,
'resource_state': self.record['state'], 'desktop_released':
self.record.get('release_results', {}).get('desktop_released') is True,
'business_uncertainty_preserved': True})
self.desktop = self.cad = None
self._active_evidence = None
self._active_attempts = []
self._close_failure = None
self._cad_close_error = None
return True
@@ -305,9 +305,9 @@ def pending_supervisors(root):
def reconcile_supervisors(root):
"""Recheck orphan ownership only; this never launches or replays a job."""
from .batch_resources import _process_gone, pending_resources, write_record
from .batch_resources import _process_gone, blocking_resources, write_record
from .read_recovery import process_tree_exited
physical = pending_resources(root)
physical = blocking_resources(root)
for record in pending_supervisors(root):
owner = (_LIVE_SUPERVISORS.get(canonical_path(record['record_path']))
or _ACTIVE_SUPERVISORS.get(canonical_path(record['record_path'])))
@@ -348,21 +348,26 @@ def _environment():
def _finish_response(root, journal, response, context):
"""A supervisor exit may release its last USER reference. Never repeat work."""
from .batch_resources import cleanup_pending_resources, write_record
from .batch_resources import cleanup_pending_resources, resource_blockers, write_record
from .supervisor_result import validated_outcome, resource_pending_error, application_shutdown_error
result, error, primary = validated_outcome(journal['operation'], response)
pending = cleanup_pending_resources(root, logging_context=context)
related = tuple(item for item in pending if item.get('session_id') == journal['batch_id'] or not item.get('session_id'))
journal['state'] = 'blocked' if related else 'finished'
journal['outcome'] = ('completed_resources_pending' if related and response.get('business_succeeded')
blockers = resource_blockers(related)
warnings = tuple(item for item in related if item not in blockers)
journal['state'] = 'blocked' if blockers else 'finished'
journal['outcome'] = ('completed_resources_pending' if blockers and response.get('business_succeeded')
else response.get('status', 'interrupted'))
journal['resource_recheck'] = {'pending': bool(related), 'checked_at': time.time()}
journal['resource_recheck'] = {'pending': bool(blockers), 'warnings': list(warnings),
'desktop_released': not related, 'owned_writers_exited': not blockers, 'checked_at': time.time()}
write_record(Path(root) / 'batch-resources' / journal['batch_id'] / 'supervisor.json', journal)
if related:
if blockers:
raise resource_pending_error(journal['operation'], result, error, primary)
for failure in (error, primary):
if failure is not None:
failure.resources_released = True
failure.resources_released = not warnings
failure.owned_writers_exited = True
failure.resource_warnings = warnings
resource_path = internal_path(Path(root) / 'batch-resources' / journal['batch_id'] / 'session.json')
def preserve_shutdown_failure(detail):
try:
@@ -373,7 +378,9 @@ def _finish_response(root, journal, response, context):
journal['shutdown_record_error'] = str(record_error)
def shutdown_failure():
failure = application_shutdown_error(result, error, primary)
failure.resources_released = True
failure.resources_released = not warnings
failure.owned_writers_exited = True
failure.resource_warnings = warnings
return failure
try:
resource = json.loads(resource_path.read_text(encoding='utf-8'))
@@ -558,6 +565,7 @@ def run_supervised(service, operation, arguments, *, stop_event=None, on_progres
on_phase(*payload)
elif value['type'] == 'state':
state = decode(value['payload'])
service._supervisor_state = state
business_watch.observe('state', state, time.monotonic())
service._task_queue_feedback = getattr(service, '_workspace_queue_context', None) or state['queue']
service._batch_feedback = state['batch']
@@ -578,6 +586,7 @@ def run_supervised(service, operation, arguments, *, stop_event=None, on_progres
service._task_queue_feedback = None
service._batch_feedback = None
service._supervisor_waiting = False
service._supervisor_state = None
service._supervisor_business_stalled = False
service._supervisor_dialog_state = None
try:
@@ -616,10 +625,10 @@ def run_supervised(service, operation, arguments, *, stop_event=None, on_progres
exit_code=getattr(process, 'returncode', None),
stdout_chars=len(stdout_tail), stderr_chars=len(stderr_tail))
if force_accepted:
from .batch_resources import cleanup_pending_resources
from .batch_resources import cleanup_pending_resources, resource_blockers
try:
pending = cleanup_pending_resources(root, logging_context=service._logging)
if pending:
if resource_blockers(pending):
ownership_cleanup_error = TaskCenterError('batch_resources_pending', '强制结束后的资源仍待清理。')
except Exception as error:
ownership_cleanup_error = error
@@ -628,7 +637,8 @@ def run_supervised(service, operation, arguments, *, stop_event=None, on_progres
failure = TaskCenterError('mentor_release_required' if automatic else 'user_forced_stop',
'执行失败,已自动收尾本批后台进程;已有结果保留,当前结果待核验。' if automatic else
'已按确认强制结束;已保存的执行证据保留,当前结果待核验。请先核验恢复,不要重复替换。')
failure.resources_released = ownership_cleanup_error is None
failure.resources_released = ownership_cleanup_error is None and not pending
failure.owned_writers_exited = ownership_cleanup_error is None
failure.primary_error = transport_error
if ownership_cleanup_error is not None:
journal.update(force_state='confirmation_failed', force_cleanup_error=str(ownership_cleanup_error))
@@ -688,7 +698,7 @@ def run_supervised(service, operation, arguments, *, stop_event=None, on_progres
raise failure from protection_error
# Recheck physical resources even when final business results are
# missing; this never authorizes repeating the interrupted job.
from .batch_resources import cleanup_pending_resources
from .batch_resources import cleanup_pending_resources, resource_blockers
outstanding = cleanup_pending_resources(root, logging_context=service._logging)
failure = cause if isinstance(cause, TaskCenterError) else TaskCenterError(
'supervisor_interrupted', '批次监督进程未留下可信结果;保留保护记录,不自动重试。' +
@@ -697,6 +707,8 @@ def run_supervised(service, operation, arguments, *, stop_event=None, on_progres
from .read_recovery import process_tree_exited
failure.resources_released = (not outstanding and ownership_cleanup_error is None
and (process is None or process_tree_exited(process)))
failure.owned_writers_exited = (not resource_blockers(outstanding) and ownership_cleanup_error is None
and (process is None or process_tree_exited(process)))
raise failure from (transport_error or error)
if transport_error is not None:
if response['payload']['business_succeeded']:
@@ -705,9 +717,13 @@ def run_supervised(service, operation, arguments, *, stop_event=None, on_progres
else:
_mark_interrupted(root, journal, transport_error)
failure = resource_pending_error(operation, decoded_result, transport_error, decoded_primary)
from .batch_resources import pending_resources
from .batch_resources import pending_resources, resource_blockers
from .read_recovery import process_tree_exited
failure.resources_released = (ownership_cleanup_error is None and not pending_resources(root)
pending = pending_resources(root)
failure.resources_released = (ownership_cleanup_error is None and not pending
and process is not None and process_tree_exited(process))
failure.owned_writers_exited = (ownership_cleanup_error is None and not resource_blockers(pending)
and process is not None and process_tree_exited(process))
failure.resource_warnings = tuple(row for row in pending if row not in resource_blockers(pending))
raise failure
return _finish_response(root, journal, response['payload'], service._logging)
@@ -14,6 +14,14 @@ from uuid import UUID
from .path_access import internal_path, canonical_path
from .supervisor_protocol import VERSION, OPERATIONS, valid_journal, encode, decode, encode_error
def _persist_result(directory, envelope):
from .batch_resources import write_record
from .operation_diagnostics import operation_span
with operation_span(directory, 'result_persistence', item_id=envelope['attempt_id']):
write_record(directory / 'result.json', envelope)
def _output_stream():
if sys.stdout is not None:
return sys.stdout, False
@@ -85,7 +93,7 @@ def consume_manual_dialog_actions(directory, runtime, batch_id, attempt_id):
write_record(claimed.with_suffix('.result'), result)
def _wait_for_owned_writes(runtime, control):
def _wait_for_owned_writes(runtime, control) -> Exception | None:
"""Process exit would close kill-on-close Jobs, so it is also a write boundary."""
if runtime.read_only:
return
@@ -103,15 +111,21 @@ def _wait_for_owned_writes(runtime, control):
return False
return all(_worker_tree_exited(entry['process'], entry['owned'])
for entry in tuple(RETAINED_WORKERS.values()) if entry['root'].is_relative_to(runtime.root))
if not exited():
try:
wait_for_resolution(control, runtime.path.parent,
'supervisor exit still has owned write processes', resolved=exited, release_on_failure=True)
except MentorReleaseRequired:
cleanup_retained_workers(runtime.root, force=True)
cleanup_retained_workers(runtime.root)
if runtime.desktop is not None or runtime.cad is not None:
runtime.close(reason='resource_recheck')
try:
if not exited():
try:
wait_for_resolution(control, runtime.path.parent,
'supervisor exit still has owned write processes', resolved=exited, release_on_failure=True)
except MentorReleaseRequired:
cleanup_retained_workers(runtime.root, force=True)
cleanup_retained_workers(runtime.root)
if runtime.desktop is not None or runtime.cad is not None:
runtime.close(reason='resource_recheck')
except Exception as error:
# Desktop references may remain after writers exit; publish that failure.
if not exited():
raise
return error
def run(request_path):
@@ -154,7 +168,7 @@ def run(request_path):
'generation': max(1, runtime.generation), 'sequence': sequence,
'type': kind, 'payload': payload}
if persist:
write_record(directory / 'result.json', envelope)
_persist_result(directory, envelope)
try:
output.write(json.dumps(envelope, ensure_ascii=True, allow_nan=False) + '\n')
output.flush()
@@ -175,9 +189,13 @@ def run(request_path):
emit('heartbeat')
feedback = getattr(service, '_task_queue_feedback', None)
emit('state', encode({'phase': runtime.record['state'], 'queue': feedback,
'release_state': runtime.record.get('release_state'),
'batch': service.batch_feedback(),
'active_project': runtime.record.get('active_project'),
'business_progress': runtime.business_progress(),
'stopping_reason': runtime.record.get('stopping_reason') or getattr(runtime.stop_event, 'stopping_reason', ''),
'wait_reason': runtime.record.get('wait_reason') or getattr(runtime.stop_event, 'wait_reason', ''),
'primary_error': runtime.record.get('primary_error') or getattr(runtime.stop_event, 'primary_error', ''),
'dialog_supervision': runtime.dialog_state(),
'dialog_binding': {'batch_id': batch_id, 'attempt_id': attempt_id,
'generation': runtime.generation, 'session_token': getattr(runtime.cad, 'token', None)},
@@ -249,7 +267,15 @@ def run(request_path):
context.report_exception('batch_supervisor', error)
response.update(status='error', error=encode_error(error))
finally:
_wait_for_owned_writes(runtime, stop)
cleanup_error = _wait_for_owned_writes(runtime, stop)
if cleanup_error is not None:
context.report_exception('batch_supervisor_resource_recheck', cleanup_error)
response['status'] = 'error'
response.setdefault('error', encode_error(cleanup_error))
if runtime.record.get('state') == 'isolated':
response['resource_warning'] = {'state': 'isolated', 'blocking': False,
'desktop': runtime.record['desktop'], 'desktop_released': False,
'owned_writers_exited': True}
evidence.try_append('worker_closing', status=response.get('status'), business_succeeded=response.get('business_succeeded'))
finished.set()
thread.join(3)
@@ -40,8 +40,9 @@ def retained_bom_rows(pending, existing_rows):
candidates = tuple(index for identity in dict.fromkeys(((record.old.casefold(), ref.casefold()),
(record.new.casefold(), ref.casefold()))) for index in by_identity.get(identity, ())
if (not record.schematic or existing_rows[index].target.schematic_name.casefold() == record.schematic.casefold())
and (not record.sheet or existing_rows[index].target.sheet_number.casefold() == record.sheet.casefold())
and (not record.uid or existing_rows[index].target.uid == record.uid))
and (not record.sheet or existing_rows[index].target.sheet_number.casefold() == record.sheet.casefold()))
if len(candidates) > 1 and record.uid:
candidates = tuple(index for index in candidates if existing_rows[index].target.uid == record.uid)
if len(candidates) > 1:
raise TaskCenterError('bom_scope_ambiguous', f'位号 {ref} 无法唯一匹配已保存位号;请填写原理图、图页或 UID:'
f'{request.source_file} 第 {request.source_row} 行')
@@ -25,7 +25,7 @@ _HEADER_HELP = (
'位号,必填。一行可填 R1,R2,R3,支持逗号、分号、空格或顿号分隔。',
'原理图名称,可选。位号重复时用于精确匹配。',
'图页,可选。位号跨页重复时应填写图页或 UID。',
'器件 UID,可选;填写时必须与采集结果一致。',
'器件 UID,可选。工程、图页和位号已唯一匹配时忽略;仅用于区分仍有多个候选的位号。',
'是否替换:TRUE/FALSE 或 是/否;留空默认替换。未列出的位号保留。',
'自定义 Value,可选;留空沿用工具原有 Value 策略。',
'同步 NC:TRUE/FALSE 或 是/否;留空默认开启。',
@@ -209,8 +209,9 @@ class BomMatcher:
for key, target in self.by_identity.get(identity, ())
if (not record.project_path or self.identities.project(record.project_path) == key[0])
and (not record.schematic or target.schematic_name.casefold() == record.schematic.casefold())
and (not record.sheet or target.sheet_number.casefold() == record.sheet.casefold())
and (not record.uid or target.uid == record.uid)]
and (not record.sheet or target.sheet_number.casefold() == record.sheet.casefold())]
if len(candidates) > 1 and record.uid:
candidates = [(key, target) for key, target in candidates if target.uid == record.uid]
if len(candidates) != 1:
raise TaskCenterError('bom_import', f'位号 {refdes} 无法唯一匹配当前采集清单')
key, target = candidates[0]
@@ -9,16 +9,20 @@ import stat
import zipfile
from .path_access import canonical_path, internal_path
from .operation_diagnostics import summarize_operations
MAX_FILE_BYTES = 4 * 1024 * 1024
MAX_BUNDLE_BYTES = 32 * 1024 * 1024
MAX_FILES = 512
_BATCH_NAMES = {'session.json', 'supervisor.json', 'request.json', 'result.json',
'business-result.json', 'supervisor.log', 'resource-release.log', 'cancel.json',
'process.json', 'lifecycle.jsonl', 'owner-loss.json', 'claim.json', 'force-request.json'}
'process.json', 'lifecycle.jsonl', 'owner-loss.json', 'claim.json', 'force-request.json',
'operation-events.jsonl'}
_QUICK_NAMES = {'private-desktop.json', 'checkpoint.json', 'status.json',
'uncertain.flag', 'resolved-uncertain.flag', 'protection.json', 'cleanup.json',
'project-session.json', 'project-checkpoint.json', 'project-read-release.json', 'status.tsv'}
'project-session.json', 'project-checkpoint.json', 'project-read-release.json', 'status.tsv',
'worker-progress.json', 'quick-part-replace.log', 'native-inventory.json', 'operation-events.jsonl',
'recovery-commit.json', 'retry-preparation.json'}
_SECRET_KEY = re.compile(r'(?:password|passwd|(?:client[_-]?)?secret|credentials?|authorization|api[_-]?key|access[_-]?token|refresh[_-]?token|token)', re.I)
_USER_KEY = re.compile(r'(?:user(?:[_-]?name)?|login[_-]?name)', re.I)
_SECRET_TEXT = re.compile(r'''(?ix)(\b(?:password|passwd|(?:client[_-]?)?secret|credentials?|authorization|api[_-]?key|access[_-]?token|refresh[_-]?token|token)\b["']?\s*[=:]\s*)(?:"[^"\r\n]*"|'[^'\r\n]*'|[^\s,;}]+)''')
@@ -52,6 +56,15 @@ def _scrub_json(value):
def _allowed(parts):
if parts[0] == 'execution-proof' or (len(parts) > 3 and parts[0] == 'tasks' and parts[2] == 'execution-proof'):
return (parts[-1] == 'completion.json'
or bool(re.fullmatch(r'project-(?:commit|pause)-[a-f0-9]{32}\.tsv', parts[-1]))
or bool(re.fullmatch(r'(?:post-state|value-readback)-[A-Za-z0-9_.-]+\.(?:tsv|json)', parts[-1])))
if (len(parts) >= 5 and parts[:1] == ('tasks',) and parts[2] == 'netlist-checks'):
return parts[-1] in {'verification.json', 'netlist.json', 'capture.json', 'export-cache.json'}
if (any(re.fullmatch(r'native-inventory-[A-Za-z0-9-]+', part) for part in parts)
and len(parts) >= 3 and parts[-2] == 'output'):
return parts[-1] in {'scope.txt', 'effective.txt'} or bool(re.fullmatch(r'icdb2ascii-(?:scope|effective)\.(?:stdout|stderr)\.txt', parts[-1], re.I))
if re.fullmatch(r'cad-dialogs-[a-f0-9]{32}\.json', parts[-1]):
return ((len(parts) == 4 and parts[0] == 'batch-resources' and re.fullmatch(r'generation-\d+', parts[2])) or
any(value in parts for value in ('quick-run', 'quickrun', 'quick-run-history')))
@@ -71,7 +84,10 @@ def _allowed(parts):
if len(parts) >= 2 and parts[0] in {'queue', 'queues', 'task-queues', 'workspace-queues'}:
return parts[-1].endswith('.json')
if len(parts) >= 2 and any(value in parts for value in ('quick-run', 'quickrun', 'quick-run-history')):
return parts[-1] in _QUICK_NAMES or bool(re.fullmatch(r'retained-worker-[a-f0-9]{32}\.json', parts[-1]))
return (parts[-1] in _QUICK_NAMES
or bool(re.fullmatch(r'retained-worker-[a-f0-9]{32}\.json', parts[-1]))
or bool(re.fullmatch(r'project-(?:commit|pause)-[a-f0-9]{32}\.tsv', parts[-1]))
or bool(re.fullmatch(r'(?:post-state|value-readback)-[A-Za-z0-9_.-]+\.(?:tsv|json)', parts[-1])))
return False
@@ -84,11 +100,13 @@ def export_diagnostic_bundle(data_root, destination: Path) -> Path:
root, destination = canonical_path(data_root), canonical_path(destination)
if destination.suffix.lower() != '.zip':
raise ValueError('diagnostic destination must have a .zip suffix')
manifest = {'version': 1, 'included': [], 'omissions': [], 'unknowns': [
'External desktop handle holders are unknown; empty window/thread lists do not prove release.',
manifest = {'version': 2, 'included': [], 'omissions': [], 'truncated': [], 'unknowns': [
'Active logs and metadata are point-in-time reads; no source CAD, libraries, BOM, database or environment configuration is included.'],
'limits': {'max_file_bytes': MAX_FILE_BYTES, 'max_bundle_bytes': MAX_BUNDLE_BYTES, 'max_files': MAX_FILES}}
total = 0
candidates = []
operations = {}
items = {}
physical = internal_path(root)
# os.walk does not descend symlinks; explicitly prune Windows junctions too.
with zipfile.ZipFile(internal_path(destination), 'x', compression=zipfile.ZIP_DEFLATED) as archive:
@@ -101,7 +119,7 @@ def export_diagnostic_bundle(data_root, destination: Path) -> Path:
info = entry.lstat()
if stat.S_ISLNK(info.st_mode) or getattr(info, 'st_file_attributes', 0) & 0x400:
manifest['omissions'].append({'path': _scrub_text(relative), 'reason': 'link_or_reparse_point'})
elif base == physical and name not in {'logs', 'batch-resources', 'attempts', 'tasks', 'queue', 'queues', 'task-queues', 'workspace-queues', 'quick-run', 'quick-run-history'}:
elif base == physical and name not in {'logs', 'batch-resources', 'attempts', 'tasks', 'queue', 'queues', 'task-queues', 'workspace-queues', 'quick-run', 'quick-run-history', 'execution-proof'}:
manifest['omissions'].append({'path': _scrub_text(relative), 'reason': 'outside_allowlist'})
else:
safe_folders.append(name)
@@ -119,28 +137,94 @@ def export_diagnostic_bundle(data_root, destination: Path) -> Path:
reason = 'link_or_reparse_point'
elif not stat.S_ISREG(info.st_mode):
reason = 'not_regular_file'
elif info.st_size > MAX_FILE_BYTES or total + info.st_size > MAX_BUNDLE_BYTES or len(manifest['included']) >= MAX_FILES:
reason = 'size_or_count_limit'
else:
with path.open('rb') as source:
payload = source.read(MAX_FILE_BYTES + 1)
if len(payload) > MAX_FILE_BYTES or b'\x00' in payload:
reason = 'binary_or_size_limit'
else:
text = payload.decode('utf-8-sig')
if relative.suffix == '.json':
text = json.dumps(_scrub_json(json.loads(text)), ensure_ascii=False, indent=2)
elif '.jsonl' in relative.name:
text = '\n'.join(json.dumps(_scrub_json(json.loads(line)), ensure_ascii=False)
for line in text.splitlines() if line.strip()) + '\n'
else:
text = _scrub_text(text)
archive.writestr(label, text)
total += len(payload)
manifest['included'].append(label)
candidates.append((info.st_mtime_ns, path, relative, label, info.st_size))
except (OSError, UnicodeError, ValueError):
reason = 'unreadable_or_invalid_text'
if reason:
manifest['omissions'].append({'path': label, 'reason': reason})
# Current failures must survive a busy workspace's file/count budget.
for _, path, relative, label, size in sorted(candidates, key=lambda row: (-row[0], row[3])):
reason = None
budget = min(MAX_FILE_BYTES, MAX_BUNDLE_BYTES - total)
try:
tail = bool(re.search(r'\.(?:log|jsonl)(?:\.\d+)?$', relative.name))
truncated = size > budget
if len(manifest['included']) >= MAX_FILES or budget <= 0 or (truncated and (not tail or budget < 64)):
reason = 'size_or_count_limit'
else:
info = path.lstat()
if stat.S_ISLNK(info.st_mode) or getattr(info, 'st_file_attributes', 0) & 0x400 or info.st_nlink != 1:
manifest['omissions'].append({'path': label, 'reason': 'link_or_reparse_point'})
continue
with path.open('rb') as source:
if truncated:
source.seek(max(0, size - budget))
source.readline(budget + 1) # Discard clipped credentials and UTF-8 lines.
payload = source.read(budget + 1)
if len(payload) > budget or b'\x00' in payload:
reason = 'binary_or_size_limit'
elif not payload and size:
reason = 'no_complete_tail_line'
else:
try:
content = payload.decode('utf-8-sig')
except UnicodeDecodeError:
if relative.name not in {'scope.txt', 'effective.txt'}:
raise
content = payload.decode('mbcs' if os.name == 'nt' else 'cp1252')
if relative.suffix == '.json':
content = json.dumps(_scrub_json(json.loads(content)), ensure_ascii=False, indent=2)
elif '.jsonl' in relative.name:
records, invalid = [], 0
for line in content.splitlines():
if not line.strip():
continue
try:
records.append(_scrub_json(json.loads(line)))
except ValueError:
invalid += 1
content = '\n'.join(json.dumps(record, ensure_ascii=False) for record in records) + '\n'
if invalid:
manifest['omissions'].append({'path': label, 'reason': 'invalid_jsonl_lines', 'lines': invalid})
else:
content = _scrub_text(content)
encoded = content.encode('utf-8')
if tail and budget >= 64 and len(encoded) > budget:
# Redaction may grow a short secret. Trim complete oldest lines again.
offset, remaining = 0, len(encoded)
for line in encoded.splitlines(keepends=True):
if remaining <= budget:
break
offset += len(line)
remaining -= len(line)
encoded = encoded[offset:]
content = encoded.decode('utf-8')
truncated = True
if '.jsonl' in relative.name:
records = [json.loads(line) for line in content.splitlines() if line.strip()]
if len(encoded) > budget:
reason = 'size_or_count_limit'
else:
archive.writestr(label, encoded)
total += len(encoded)
manifest['included'].append(label)
if truncated:
manifest['truncated'].append({'path': label, 'mode': 'tail', 'source_bytes': size})
if relative.name == 'operation-events.jsonl':
operations[_scrub_text(relative.parent.as_posix())] = summarize_operations(records)
elif relative.name == 'status.tsv':
rows = [line.split('\t') for line in content.splitlines() if line.strip()]
rows = [row for row in rows if len(row) >= 4 and row[0] != 'job_id']
if rows:
saved = [row[0] for row in rows if row[1] in {'saved_ok', 'restored_ok'}]
items[_scrub_text(relative.parent.as_posix())] = {
'last_successful_item': saved[-1] if saved else None,
'latest_item': {'job_id': rows[-1][0], 'state': rows[-1][1]}}
except (OSError, UnicodeError, ValueError):
reason = 'unreadable_or_invalid_text'
if reason:
manifest['omissions'].append({'path': label, 'reason': reason})
archive.writestr('closure-summary.json', json.dumps({'operations': operations, 'items': items}, ensure_ascii=False, indent=2))
archive.writestr('manifest.json', json.dumps(manifest, ensure_ascii=False, indent=2))
return destination
@@ -14,25 +14,43 @@ class ExecutionFeedback:
stage: str = ''
stage_seconds: float = 0.0
waiting_for_safe_completion: bool = False
stopping_reason: str = ''
wait_reason: str = ''
primary_error: str = ''
def wait_reason_for_stage(stage):
return {'save_and_close': 'save', 'verify_saved_project': 'readback',
'native_snapshot_read': 'readback', 'capture_semantic_state': 'readback',
'private_shutdown': 'desktop_release',
'awaiting_reconciliation': 'reconciliation', 'mentor_exit': 'mentor_exit',
'desktop_release': 'desktop_release'}.get(stage,
'native_call' if stage in {'native_replace', 'restore_attributes',
'restore_attribute_fields', 'verify_restoration', 'verify_attribute_fields'} else '')
def read_feedback(task_id, position, total, message, root: Path, started_after: float):
private_state = ''
private_seconds = 0.0
private = root / 'quick-run' / 'private-desktop.json'
try:
modified = private.stat().st_mtime
if modified >= started_after:
state = json.loads(private.read_text(encoding='utf-8')).get('state')
if state in {'starting', 'closing'}:
return ExecutionFeedback(task_id, position, total, message,
'private_startup' if state == 'starting' else 'private_shutdown', max(0, time.time() - modified))
private_state = 'private_startup' if state == 'starting' else 'private_shutdown'
private_seconds = max(0, time.time() - modified)
except (OSError, ValueError, TypeError, AttributeError):
pass
path = root / 'quick-run' / 'worker-progress.json'
stage, seconds, waiting = '', 0.0, False
record = {}
try:
modified = path.stat().st_mtime
if modified >= started_after:
record = json.loads(path.read_text(encoding='utf-8'))
if not isinstance(record, dict):
record = {}
waiting = record.get('waiting_for_safe_completion') is True
candidate = record.get('stage', '')
if candidate and candidate != 'finished':
@@ -40,4 +58,14 @@ def read_feedback(task_id, position, total, message, root: Path, started_after:
stage = candidate
except (OSError, ValueError, TypeError, AttributeError):
pass # Display feedback is optional; it never changes execution state.
return ExecutionFeedback(task_id, position, total, message, stage, seconds, waiting)
if private_state:
stage, seconds = private_state, private_seconds
stopping_reason = record.get('stopping_reason', '')
if not stopping_reason and record.get('stop_requested'):
stopping_reason = 'inactivity'
wait_reason = record.get('wait_reason') or wait_reason_for_stage(stage)
if private_state and wait_reason not in {'mentor_exit', 'desktop_release'}:
wait_reason = wait_reason_for_stage(stage)
return ExecutionFeedback(task_id, position, total, message, stage, seconds, waiting,
stopping_reason, wait_reason,
record.get('primary_error') or record.get('resolution_error') or '')
@@ -33,6 +33,11 @@ def create_execution_schema(connection):
connection.execute('''CREATE TABLE IF NOT EXISTS execution_target_claims (
task_id TEXT NOT NULL, round_id TEXT NOT NULL, target_key TEXT NOT NULL,
operation_key TEXT NOT NULL, PRIMARY KEY(task_id,round_id,target_key))''')
connection.execute('''CREATE TABLE IF NOT EXISTS execution_project_commits (
task_id TEXT NOT NULL, round_id TEXT NOT NULL, session_token TEXT NOT NULL,
project_path TEXT NOT NULL, evidence_root TEXT NOT NULL, state TEXT NOT NULL,
jobs_json TEXT NOT NULL, proof_json TEXT,
PRIMARY KEY(task_id,round_id,session_token))''')
def _json(value):
@@ -115,7 +120,7 @@ class ExecutionJournal:
current=self._round(task_id)
if current:
unsafe=self.store._connection.execute('''SELECT 1 FROM execution_operations
WHERE task_id=? AND round_id=? AND state IN ('prepared','uncertain') LIMIT 1''',
WHERE task_id=? AND round_id=? AND state IN ('prepared','uncertain','pending_save') LIMIT 1''',
(task_id,current['round_id'])).fetchone()
if unsafe:raise TaskCenterError('quick_run_uncertain','原执行项尚待核验,不能准备重试。')
self.store._connection.execute('DELETE FROM execution_retry_selections WHERE task_id=?',(task_id,))
@@ -154,7 +159,7 @@ class ExecutionJournal:
(task_id, current['round_id']))}
for key in json.loads(current['plan_json']):
record = records.get(key)
if record is None or record['state'] in {'prepared', 'uncertain'}:return False
if record is None or record['state'] in {'prepared', 'uncertain', 'pending_save'}:return False
if record['state'] == 'pending':
command = json.loads(record['command_json'])
return bool(self.can_continue_component(task_id, command))
@@ -430,7 +435,7 @@ class ExecutionJournal:
with self.store._lock:
current=self._round(task_id)
return bool(current and self.store._connection.execute('''SELECT 1 FROM execution_operations
WHERE task_id=? AND round_id=? AND state='prepared' LIMIT 1''',
WHERE task_id=? AND round_id=? AND state IN ('prepared','pending_save') LIMIT 1''',
(task_id,current['round_id'])).fetchone())
def unresolved_jobs(self,task_id,rows):
@@ -438,7 +443,7 @@ class ExecutionJournal:
current=self._round(task_id)
if current is None:return ()
keys={record[0] for record in self.store._connection.execute('''SELECT operation_key
FROM execution_operations WHERE task_id=? AND round_id=? AND state IN ('prepared','uncertain')''',
FROM execution_operations WHERE task_id=? AND round_id=? AND state IN ('prepared','uncertain','pending_save')''',
(task_id,current['round_id']))}
return tuple(row['job_id'] for row in rows if _hash(command_identity(row)) in keys)
@@ -506,6 +511,8 @@ class ExecutionJournal:
elif record['state']=='failed':
restored[row['job_id']]=tuple(json.loads(record['receipt_json'])[name] for name in ('native_state','code','message'))
continue
elif record['state']=='pending_save':
raise TaskCenterError('quick_run_uncertain','工程替换尚待统一保存回读,禁止重放;请先核验工程提交记录。')
outcome=latest.get(row['job_id'])
if outcome and outcome[0] in _TERMINAL:
record=connection.execute('''SELECT * FROM execution_operations
@@ -546,6 +553,136 @@ class ExecutionJournal:
WHERE task_id=? AND round_id=? AND operation_key=?''',(identifier,generation,task_id,current['round_id'],key))
return JournalAttempt(task_id,current['round_id'],key,identifier,generation,row['job_id'])
def stage_project_result(self, attempt, row, *, session_token, root, proof):
"""Retain an applied operation without granting a saved receipt or retry."""
from .execution_proof import proof_intact
if not session_token or not proof_intact(proof):
raise TaskCenterError('quick_run_uncertain', '工程暂存证据不完整。')
with self.store.transaction():
connection = self.store._connection
record = connection.execute('''SELECT * FROM execution_operations
WHERE task_id=? AND round_id=? AND operation_key=?''',
(attempt.task_id, attempt.round_id, attempt.operation_key)).fetchone()
if (record is None or record['state'] != 'prepared' or record['attempt_id'] != attempt.attempt_id
or record['generation'] != attempt.generation or _hash(command_identity(row)) != attempt.operation_key):
raise TaskCenterError('quick_run_uncertain', '工程暂存结果不属于当前执行声明。')
project = command_identity(row)['project_path']
commit = connection.execute('''SELECT * FROM execution_project_commits
WHERE task_id=? AND round_id=? AND session_token=?''',
(attempt.task_id, attempt.round_id, session_token)).fetchone()
if commit and (commit['state'] != 'pending_save' or commit['project_path'] != project
or commit['evidence_root'] != str(canonical_path(root))):
raise TaskCenterError('quick_run_uncertain', '工程提交身份发生变化。')
jobs = json.loads(commit['jobs_json']) if commit else []
if row['job_id'] in jobs:
raise TaskCenterError('quick_run_uncertain', '工程暂存操作重复。')
jobs.append(row['job_id'])
connection.execute('''INSERT INTO execution_project_commits VALUES (?,?,?,?,?, 'pending_save',?,NULL)
ON CONFLICT(task_id,round_id,session_token) DO UPDATE SET jobs_json=excluded.jobs_json''',
(attempt.task_id, attempt.round_id, session_token, project, str(canonical_path(root)), _json(jobs)))
receipt = dict(native_state='pending_save', code='session_staged', message='替换已暂存,等待工程统一保存回读。',
job_id=row['job_id'], session_token=session_token, row=row, proof=proof,
operation_attempt=[attempt.operation_key, attempt.attempt_id, attempt.generation])
connection.execute('''UPDATE execution_operations SET state='pending_save',receipt_json=?
WHERE task_id=? AND round_id=? AND operation_key=?''',
(_json(receipt), attempt.task_id, attempt.round_id, attempt.operation_key))
def pending_project_commits(self, task_id):
with self.store._lock:
current = self._round(task_id)
if not current:return ()
return tuple(dict(row) for row in self.store._connection.execute('''SELECT * FROM execution_project_commits
WHERE task_id=? AND round_id=? AND state='pending_save' ''', (task_id, current['round_id'])))
def project_commit_records(self, task_id):
with self.store._lock:
current = self._round(task_id)
return tuple(dict(row) for row in self.store._connection.execute('''SELECT * FROM execution_project_commits
WHERE task_id=? AND round_id=?''', (task_id, current['round_id']))) if current else ()
def project_commit_attempts(self, task_id, session_token):
with self.store._lock:
current = self._round(task_id)
if not current:return {}
result = {}
for row in self.store._connection.execute('''SELECT * FROM execution_operations
WHERE task_id=? AND round_id=?''', (task_id, current['round_id'])):
receipt = json.loads(row['receipt_json'] or '{}')
token = receipt.get('session_token') or receipt.get('proof', {}).get('project_commit', {}).get('session_token')
if token == session_token:
actual = [row['operation_key'], row['attempt_id'], row['generation']]
original = receipt.get('operation_attempt') or receipt.get('proof', {}).get('project_commit', {}).get('operation_attempts', {}).get(receipt['job_id'])
if original is not None and original != actual:
raise TaskCenterError('quick_run_uncertain', '原工程暂存尝试与当前执行记录不同。')
result[receipt['job_id']] = actual
return result
def project_commit_recovery_attempts(self, task_id):
"""Include actual attempts outside the frozen command rows as well."""
with self.store._lock:
current = self._round(task_id)
if not current:return ()
return tuple([row['operation_key'], row['attempt_id'], row['generation']]
for row in self.store._connection.execute('''SELECT operation_key,attempt_id,generation
FROM execution_operations WHERE task_id=? AND round_id=?
AND (attempt_id IS NOT NULL OR generation>0)''', (task_id, current['round_id'])))
def bind_project_commit_proof(self, task_id, session_token, proof):
with self.store.transaction():
current = self._round(task_id)
self.store._connection.execute('''UPDATE execution_project_commits SET proof_json=?
WHERE task_id=? AND round_id=? AND session_token=? AND state='pending_save' AND proof_json IS NULL''',
(_json(proof), task_id, current['round_id'], session_token))
def staged_project_rows(self, task_id, session_token):
with self.store._lock:
current = self._round(task_id)
if not current:return ()
records = self.store._connection.execute('''SELECT receipt_json FROM execution_operations
WHERE task_id=? AND round_id=? AND state='pending_save' ''', (task_id, current['round_id']))
return tuple(receipt for record in records
if (receipt := json.loads(record[0])).get('session_token') == session_token)
def commit_project_results(self, task_id, session_token, proofs, commit_proof):
"""Commit a whole staged prefix and its history in one SQL transaction."""
from .execution_proof import proof_intact
with self.store.transaction():
connection = self.store._connection
current = self._round(task_id)
commit = connection.execute('''SELECT * FROM execution_project_commits
WHERE task_id=? AND round_id=? AND session_token=?''', (task_id, current['round_id'], session_token)).fetchone() if current else None
if commit is None:
raise TaskCenterError('quick_run_uncertain', '缺少原工程暂存提交记录。')
if commit['state'] == 'verified':return
jobs = json.loads(commit['jobs_json'])
if (commit_proof.get('session_token') != session_token
or str(canonical_path(commit_proof.get('project_path', ''))).casefold() != commit['project_path']
or commit_proof.get('jobs') != jobs or set(proofs) != set(jobs)
or not commit_proof.get('files') or not proof_intact(commit_proof)
or any(not proof.get('files') or not proof_intact(proof) for proof in proofs.values())):
raise TaskCenterError('quick_run_uncertain', '工程保存回读证明未覆盖全部暂存项。')
records = connection.execute('''SELECT * FROM execution_operations
WHERE task_id=? AND round_id=? AND state='pending_save' ''', (task_id, current['round_id'])).fetchall()
records = [record for record in records if json.loads(record['receipt_json']).get('session_token') == session_token]
if {json.loads(record['receipt_json'])['job_id'] for record in records} != set(jobs):
raise TaskCenterError('quick_run_uncertain', '工程暂存前缀与提交证明不一致。')
for record in records:
receipt = json.loads(record['receipt_json'])
job = receipt['job_id']
proof = dict(proofs[job])
if proof.get('native_attempt') != receipt.get('proof', {}).get('native_attempt'):
raise TaskCenterError('quick_run_uncertain', '保存回读与原暂存原生尝试不一致。')
proof['project_commit'] = commit_proof
proof['files'] = list(proof['files']) + list(commit_proof['files'])
attempt = JournalAttempt(task_id, current['round_id'], record['operation_key'],
record['attempt_id'], record['generation'], job)
connection.execute('''UPDATE execution_operations SET state='prepared'
WHERE task_id=? AND round_id=? AND operation_key=?''', (task_id, current['round_id'], record['operation_key']))
native_state = commit_proof.get('outcomes', {}).get(job, 'saved_ok')
self.finish(attempt, native_state, 'project_committed', '工程统一保存、关闭重开及回读校验通过。', proof=proof)
connection.execute('''UPDATE execution_project_commits SET state='verified',proof_json=?
WHERE task_id=? AND round_id=? AND session_token=?''', (_json(commit_proof), task_id, current['round_id'], session_token))
def finish(self,attempt,native_state,code,message,*,progress=None,proof=None):
if native_state not in _TERMINAL:raise TaskCenterError('quick_run_uncertain','不能提交未知的执行结果。')
state=({'saved_ok':'verified','no_match':'no_match','restored_ok':'restored',
@@ -682,7 +819,7 @@ class ExecutionJournal:
from .execution_proof import proof_intact
if not proof_intact(receipt.get('proof',{})):return True
if not outcome or tuple(outcome[:2])!=(receipt['native_state'],receipt['code']):return True
elif record['state']=='uncertain':return True
elif record['state'] in {'uncertain','pending_save'}:return True
elif record['state']=='prepared' and (not outcome or outcome[0] not in _TERMINAL):return True
return False
@@ -33,7 +33,7 @@ def capture_result(data_root,root,row,outcome,*,strict=False,persist=True):
checked_manifest = manifest.read_bytes()
paths.append(manifest)
proof['inventory_empty'] = True
if strict and state in {'saved_ok','no_match'} and code not in {'batch_closed','session_verified'}:
if strict and state in {'saved_ok','no_match'} and code not in {'batch_closed','session_verified','project_committed'}:
raise ValueError('save/close was not verified')
checked={}
if row.get('execution_unit') == 'inventory_empty':
@@ -55,7 +55,7 @@ def capture_result(data_root,root,row,outcome,*,strict=False,persist=True):
paths.append(baseline)
elif strict and state in {'saved_ok','no_match','restored_ok','recovery_readback'}:
raise ValueError('missing verified semantic state')
if state=='saved_ok' or (state=='restored_ok' and code in {'recovery_verified','recovery_saved_verified'}):
if state=='saved_ok' or (state=='pending_save' and code == 'session_staged') or (state=='restored_ok' and code in {'recovery_verified','recovery_saved_verified'}):
snapshot=internal_path(root/f'snapshot-{row["job_id"]}.tsv')
report_path=Path(str(snapshot)+'.native.json')
if strict or snapshot.is_file() or report_path.is_file():
@@ -101,6 +101,59 @@ def capture_result(data_root,root,row,outcome,*,strict=False,persist=True):
raise TaskCenterError('quick_run_uncertain','原生回执、尝试身份或工程语义基线无法核验,已停止以避免重复替换。') from error
def capture_project_commit(data_root, root, commit):
"""Bind a durable project marker to the exact staged prefix before SQL commit."""
from .quick_run import _decode_vbs_field
try:
path = internal_path(Path(root) / f'project-commit-{commit["session_token"]}.tsv')
data = path.read_bytes()
lines = data.decode('utf-8-sig').splitlines()
token, project, state = lines[0].split('\t')
outcomes = [line.split('\t') for line in lines[1:] if line]
jobs = json.loads(commit['jobs_json'])
if (token != commit['session_token'] or state != 'project_verified'
or str(canonical_path(_decode_vbs_field(project))).casefold() != commit['project_path']
or [outcome[0] for outcome in outcomes] != jobs
or any(len(outcome) != 2 or outcome[1] not in {'saved_ok', 'no_match'} for outcome in outcomes)):
raise ValueError('project commit does not cover staged prefix')
destination = internal_path(data_root / 'execution-proof' / uuid4().hex)
destination.mkdir(parents=True, exist_ok=False)
target = destination / path.name
with target.open('xb') as stream:
stream.write(data); stream.flush(); os.fsync(stream.fileno())
if path.read_bytes() != data:
raise ValueError('project marker changed during binding')
return dict(session_token=token, project_path=commit['project_path'], jobs=jobs,
outcomes=dict(outcomes), files=[dict(path=str(canonical_path(target)), sha256=hashlib.sha256(data).hexdigest())])
except (OSError, UnicodeError, ValueError, KeyError, TypeError, IndexError) as error:
raise TaskCenterError('quick_run_uncertain', '工程统一保存或回读尚未得到完整证明,保留全部暂存项待核验。') from error
def bind_project_commit_source(proof, data_root, root, project):
"""Capture saved bytes only at the completed native-close callback."""
from .project_session_health import _source_parameters, _source_digest
from .project_isolation import write_sources
value = dict(proof, **_source_parameters(Path(project), data_root))
value['source_sha256'] = _source_digest(value)
value['command_sha256'] = hashlib.sha256(internal_path(root / 'command.tsv').read_bytes()).hexdigest()
value['saved_sources'] = {source: hashlib.sha256(internal_path(source).read_bytes()).hexdigest()
for source in write_sources(project)}
if not value['source_sha256']:
raise TaskCenterError('quick_run_uncertain', '已保存工程源文件无法稳定绑定,保留待核验。')
return value
def validate_project_commit_source(proof, root):
from .project_session_health import _source_digest
try:
return (proof.get('source_sha256') and _source_digest(proof) == proof['source_sha256']
and proof.get('command_sha256') == hashlib.sha256(internal_path(root / 'command.tsv').read_bytes()).hexdigest()
and bool(proof.get('saved_sources')) and all(
hashlib.sha256(internal_path(source).read_bytes()).hexdigest() == digest
for source, digest in proof['saved_sources'].items()) and proof_intact(proof))
except (OSError, KeyError, TypeError, ValueError):return False
def capture_recovery(data_root,audit,row,outcome,*,strict=False):
"""The recovery audit proves readback; it never asserts a second replacement."""
worker=audit/'worker'
@@ -689,6 +689,7 @@ class PrivateDesktop:
close_pending()
except Exception as error:
errors.append(error)
self.cleanup_record['owned_cleanup_complete'] = not errors and not self._processes
try:
self.release_desktop()
except Exception as error:
@@ -15,6 +15,7 @@ class TaskStatus(str, Enum):
PAUSED = "paused"
COMPLETED = "completed"
FAILED = "failed"
BLOCKED = "blocked"
DELETED = "deleted"
@@ -1,6 +1,9 @@
"""Native current-sheet replacement adapter for the existing QuickPartReplace runner."""
import argparse
from collections import Counter
from contextlib import contextmanager
from contextvars import ContextVar
from functools import wraps
import faulthandler
import ctypes
import csv
@@ -15,6 +18,46 @@ import time
import traceback
_SNAPSHOT_REFERENCE_KEYS = ContextVar('snapshot_reference_keys', default={})
@contextmanager
def snapshot_identity_scope(snapshot):
"""Keep original receipt keys while resolving live COM objects by reference."""
rows = [line.split('\t') for line in Path(snapshot).read_text(encoding='utf-8-sig').splitlines()]
references, owners, spelling = {}, {}, {}
for row in rows:
if len(row) >= 3 and row[0] in {'C', 'U'}:
uid, ref = decode(row[1]), decode(row[2]).strip().casefold()
spelling[ref] = decode(row[2]).strip()
transform = tuple(decode(value) for value in row[3:7]) if row[0] == 'C' and len(row) == 8 else None
references.setdefault(ref, []).append((uid, transform))
owners.setdefault(uid, set()).add(ref)
for row in rows:
if len(row) == 3 and row[0] == 'G':
headers = [line.split('\t') for line in decode(row[2]).splitlines() if line.startswith('C\t')]
if len(headers) == 1 and len(headers[0]) == 8:
header = [decode(value) for value in headers[0]]
ref = header[2].strip().casefold()
references[ref] = [(uid, tuple(header[3:7]) if uid == header[1] else transform)
for uid, transform in references.get(ref, ())]
keys = {ref: [(uid if len(owners[uid]) == 1 else '@REFDES|' + spelling[ref], transform)
for uid, transform in values] for ref, values in references.items() if ref}
token = _SNAPSHOT_REFERENCE_KEYS.set(keys)
try:
yield
finally:
_SNAPSHOT_REFERENCE_KEYS.reset(token)
def _bound_snapshot_identity(action):
@wraps(action)
def bound(snapshot, *args, **kwargs):
with snapshot_identity_scope(snapshot):
return action(snapshot, *args, **kwargs)
return bound
def snapshot_attempt(snapshot):
"""Bind batch evidence to the actual snapshot and its confirmed queue row."""
from .batch_attempt import _bound_attempt
@@ -392,16 +435,35 @@ def components_on_sheet(app, api):
def _component_uid(component):
value = getattr(component, 'UID', None)
refdes = str(getattr(component, 'Refdes', '') or '').strip()
identities = _SNAPSHOT_REFERENCE_KEYS.get().get(refdes.casefold(), ())
if len(identities) == 1:
return identities[0][0]
if identities:
matches = [uid for uid, transform in identities if transform and _same_unit(component, transform)]
if len(matches) != 1:
raise ValueError('Snapshot physical unit association is missing or ambiguous')
return matches[0]
try:
value = getattr(component, 'UID', None)
except Exception:
value = None
uid = '' if value is None else str(value).strip()
if uid:
return uid
refdes = str(getattr(component, 'Refdes', '') or '').strip()
if not refdes:
raise ValueError("Component UID and Refdes are empty; cannot safely identify the target")
return '@REFDES|' + refdes
def _same_unit(component, transform):
point = component.GetLocation()
return (abs(float(point.X) - float(transform[0])) <= 10
and abs(float(point.Y) - float(transform[1])) <= 10
and float(component.Orientation) == float(transform[2])
and float(component.Scale) == float(transform[3]))
def check_active_context(app, metadata):
from .cad_identity import verify_application, project_file_path
verify_application(app)
@@ -418,15 +480,20 @@ def check_context(app, api, metadata, records, require_selection=True):
by_uid = {_component_uid(component): component for component in components}
if len(by_uid) != len(components):
raise ValueError("Duplicate component UID on sheet")
for record in records:
component = by_uid.get(record[1])
if component is None or str(component.Refdes).strip().casefold() != record[2].strip().casefold():
raise ValueError('Snapshot reference differs from the selected live component')
if len(metadata) == 10:
selectors = _target_selectors(metadata[9])
if set(selectors) != wanted:
selected_refs = {ref.strip().casefold() for ref, _ in selectors.values()}
if selected_refs != {record[2].strip().casefold() for record in records}:
raise ValueError("Target selector differs from snapshot components")
for uid, (refdes, old_value) in selectors.items():
component = by_uid.get(uid)
if (component is None or str(component.Refdes) != refdes or
attribute_value(component, "Part Number", required=True).casefold() != metadata[2].casefold() or
attribute_value(component, "Value") != old_value):
candidates = [component for component in components if str(component.Refdes).strip().casefold() == refdes.strip().casefold()]
if (not candidates or any(_component_uid(component) not in wanted
or attribute_value(component, "Part Number", required=True).casefold() != metadata[2].casefold()
or attribute_value(component, "Value") != old_value for component in candidates)):
raise ValueError("Selected target changed since inventory")
else:
actual = {_component_uid(component) for component in components if attribute_value(component, "Part Number", required=True).casefold() == metadata[2].casefold()}
@@ -591,20 +658,26 @@ def _promotion_phase(snapshot, report, phase):
def replacement_matches(components, records, target, require_part_label, *, library_row=None):
current_by_uid = {_component_uid(component): component for component in components}
if len(current_by_uid) != len(components):
raise ValueError('Duplicate component UID on sheet')
current_by_ref = {}
for component in components:
current_by_ref.setdefault(str(component.Refdes).strip().casefold(), []).append(component)
expected_name = expected_label = None
if library_row is not None:
if (len(library_row) < 6 or str(library_row[1]).casefold() != target.casefold()
or not str(library_row[4]).strip()):
raise ValueError('Recorded target library identity is incomplete or inconsistent')
expected_name, expected_label = str(library_row[4]), str(library_row[5])
matches = []
matches, used = [], set()
for record in records:
candidate = current_by_uid.get(record[1])
if candidate is None:
raise ValueError('Native replacement lost component UID identity')
candidates = current_by_ref.get(str(record[2]).strip().casefold(), [])
if len(candidates) > 1:
candidates = [candidate for candidate in candidates if len(record) >= 7 and _same_unit(candidate, record[3:7])]
if len(candidates) != 1:
raise ValueError('Native replacement reference is missing or ambiguous')
candidate = candidates[0]
if id(candidate) in used:
raise ValueError('Duplicate physical unit association')
used.add(id(candidate))
if attribute_value(candidate, "Part Number", required=True).casefold() != target.casefold():
return None
identity = str(candidate.Refdes).strip() or f"UID={candidate.UID}"
@@ -622,7 +695,7 @@ def replacement_matches(components, records, target, require_part_label, *, libr
if expected_name is not None and (part_name != expected_name
or (expected_label and part_label != expected_label)):
raise ValueError(f'Native replacement does not match the selected target library identity; uid={candidate.UID}')
matches.append(_component_uid(candidate))
matches.append(record[1])
if len(set(matches)) != len(records):
raise ValueError("Duplicate replacement association")
return matches
@@ -651,6 +724,7 @@ def verify_replacement_readback(snapshot, components, metadata, records, before_
return verified
@_bound_snapshot_identity
def run_native(snapshot, report):
import win32api
import win32con
@@ -854,7 +928,9 @@ def run_native(snapshot, report):
report["stage"] = "native_replace_started"
save_report(snapshot, report)
mark_snapshot_mutation(snapshot)
click(dlg_item(dialog, 1695))
from .operation_diagnostics import operation_span
with operation_span(Path(snapshot).parent, 'native_replace', item_id=Path(snapshot).stem):
click(dlg_item(dialog, 1695))
replacement_poll_count = 0
@@ -2,7 +2,7 @@
from __future__ import annotations
import os
from collections import Counter
from collections import Counter, defaultdict
from pathlib import Path
import re
@@ -38,38 +38,45 @@ def _require_report(text, mode):
def _effective_properties(text):
from .native_netlist import _is_pcb
_require_report(text, 'FLAT BLOCKS')
if len(re.findall(r'^Config .+', text, re.MULTILINE)) != 1:
raise NativeInventoryUnsupported('展开报告必须具有唯一配置')
result, attributes, paths = {}, {}, []
inconsistent_refs = set()
symbol = None
symbol = symbol_type = None
def finish():
if symbol is None:
return
uid, ref = symbol
if not _UID.fullmatch(uid):
if attributes.get('Part Number'):
raise NativeInventoryUnsupported('展开报告的物料缺少唯一 UID')
if symbol_type is not None and not _is_pcb({'symbol': symbol_type, 'attributes': attributes}):
return
if uid in result or paths != ['']:
raise NativeInventoryUnsupported('展开报告含重复或嵌套实例,需由 CAD 核验')
uid, ref, page = symbol
if not ref:
if attributes.get('Part Number') and symbol_type and _is_pcb({'symbol': symbol_type,'attributes':attributes}):
raise NativeInventoryUnsupported('展开报告的请求物料缺少位号')
return
unit = symbol_type.split(' type ', 1)[0] if symbol_type else ''
key = (ref.strip().casefold(), page.strip(), unit)
if key in result or paths != ['']:
raise NativeInventoryUnsupported('展开报告含重复位号或嵌套实例,无法明确定位')
if ref != attributes.get('Ref Designator', ''):
inconsistent_refs.add(uid)
result[uid] = dict(attributes)
inconsistent_refs.add(key)
result[key] = (uid, dict(attributes))
if len(result) > 100000:
raise NativeInventoryUnsupported('原生器件数量超过读取上限')
for line in text.splitlines():
if line.startswith(' Symbol '):
finish()
match = re.fullmatch(r' Symbol (.+?)\tref des ([^\t]*)\tpage [^\t]*', line)
match = re.fullmatch(r' Symbol (.*?)\tref des ([^\t]*)\tpage ([^\t]*)', line)
if match is None:
raise NativeInventoryUnsupported('展开报告含不明确的器件身份')
symbol, attributes, paths = match.groups(), {}, []
symbol, attributes, paths, symbol_type = match.groups(), {}, [], None
elif symbol is not None and line.startswith('\tPath '):
paths.append(line[6:])
elif symbol is not None and line.startswith('\tSymbol name '):
symbol_type = line[13:]
elif symbol is not None:
match = _PROPERTY.fullmatch(line)
if match:
@@ -82,39 +89,61 @@ def _effective_properties(text):
def _parse_scope(text, requested, effective, inconsistent_refs):
from .native_netlist import _is_pcb
blocks, sheets, targets = set(), set(), set()
seen_uids = set()
block, symbol, attributes = '', None, {}
physical_targets = {}
by_ref = defaultdict(list)
uid_counts = Counter(uid for uid, _ in effective.values())
unit_counts = Counter(key[:2] for key in effective)
for key in effective:
by_ref[key[0]].append(key)
block, symbol, attributes, symbol_type = '', None, {}, None
def finish():
if symbol is None:
return
uid, sheet = symbol
if not _UID.fullmatch(uid):
if attributes.get('Part Number'):
raise NativeInventoryUnsupported('全工程报告中的请求物料缺少唯一身份')
if symbol_type is not None and not _is_pcb({'symbol': symbol_type, 'attributes': attributes}):
return
if uid in seen_uids or uid not in effective:
raise NativeInventoryUnsupported('分层与展开报告的 UID 不完整或重复,需由 CAD 核验')
seen_uids.add(uid)
values = effective[uid]
uid, sheet = symbol
ref = attributes.get('Ref Designator','')
if not ref:
if attributes.get('Part Number','').casefold() in requested:
raise NativeInventoryUnsupported('全工程报告中的请求物料缺少位号')
return
unit = symbol_type.split(' type ', 1)[0] if symbol_type else ''
candidates = [key for key in by_ref[ref.strip().casefold()]
if (not key[1] or key[1] == sheet.strip())
and (not key[2] or not unit or key[2] == unit)]
if len(candidates) != 1 or candidates[0] in seen_uids:
raise NativeInventoryUnsupported('分层与展开报告的器件范围不完整或位号重复')
key = candidates[0]
seen_uids.add(key)
flat_uid, values = effective[key]
if attributes.get('Part Number') and 'Part Number' not in values:
raise NativeInventoryUnsupported('展开报告缺少物料生效属性')
if 'Value' in attributes and 'Value' not in values:
raise NativeInventoryUnsupported('展开报告缺少 Value 生效属性')
if values.get('Part Number', '').casefold() in requested:
# A library reference override on an unrelated part cannot affect
# the requested identities. Still validate every UID/config/path,
# the requested identities. Still validate every reference/config/path,
# and require CAD whenever the ambiguous component is requested.
if uid in inconsistent_refs:
if key in inconsistent_refs:
raise NativeInventoryUnsupported('展开报告的位号与属性不一致')
ref, part = values.get('Ref Designator', ''), values['Part Number']
if not ref or 'Ref Designator' not in attributes:
raise NativeInventoryUnsupported('全工程报告中的请求物料缺少位号')
target = (block, sheet, uid, ref, part, values.get('Value', ''),
multiunit = unit_counts[key[:2]] > 1
identity = uid if not multiunit and _UID.fullmatch(uid) and uid == flat_uid and uid_counts[uid] == 1 else '@REFDES|' + ref
target = (block, sheet, identity, ref, part, values.get('Value', ''),
values.get('Part Label'), values.get('Cell Name'))
if target in targets:
physical_key = block, sheet, ref
prior = physical_targets.get(physical_key)
if prior and prior[3:] != target[3:]:
raise NativeInventoryUnsupported('Conflicting effective attributes across physical symbol units')
if target in targets and not multiunit:
raise NativeInventoryUnsupported('全工程报告含重复器件身份')
targets.add(target)
physical_targets[physical_key] = target
if len(targets) > 100000:
raise NativeInventoryUnsupported('原生器件数量超过读取上限')
for line in text.splitlines():
@@ -124,14 +153,16 @@ def _parse_scope(text, requested, effective, inconsistent_refs):
if not name or name in blocks:
raise NativeInventoryUnsupported('全工程报告含不明确的原理图身份')
blocks.add(name)
block, symbol, attributes = name, None, {}
block, symbol, attributes, symbol_type = name, None, {}, None
elif line.startswith(' Symbol '):
finish()
match = re.fullmatch(r' Symbol (.+?)\tpage (.+)', line)
match = re.fullmatch(r' Symbol (.*?)\tpage (.+)', line)
if not block or match is None:
raise NativeInventoryUnsupported('全工程报告含不明确的图页身份')
symbol, attributes = match.groups(), {}
symbol, attributes, symbol_type = match.groups(), {}, None
sheets.add((block, symbol[1]))
elif symbol is not None and line.startswith('\tSymbol name '):
symbol_type = line[13:]
elif symbol is not None and line.startswith('\tProp '):
# Pin properties have two tabs and must never override a component.
match = _PROPERTY.fullmatch(line)
@@ -149,49 +180,46 @@ def _parse_scope(text, requested, effective, inconsistent_refs):
def _hierarchical_root_discovery(project, report_path, effective_path, requested, expected_root):
# ComponentTarget has no instance path: certify only globally unique root selections.
from .native_netlist import _parse_ascii, _check_hierarchy_graph, _is_pcb
text = read_native_text(report_path)
roots = re.findall(r'^Block ([^\t\r\n]+)\tRoot$', text, re.MULTILINE)
root = expected_root or (roots[0].rstrip() if len(roots) == 1 else None)
try:
components, _ = _parse_ascii(effective_path, require_connections=False)
_check_hierarchy_graph(report_path, {'components': components}, root)
_, _, expanded = _check_hierarchy_graph(report_path, {'components': components}, root, include_rows=True)
except ValueError as error:
raise NativeInventoryUnsupported(str(error)) from error
physical = {uid: row for uid, row in components.items() if _is_pcb(row)}
source_uids = Counter(row.get('source_uid', uid) for uid, row in components.items())
references = Counter(row['refdes'] for row in physical.values())
scopes, sheets, block = {}, set(), None
for line in text.splitlines():
if line.startswith('Block '):
block = line[6:].split('\t', 1)[0].rstrip()
elif line.startswith(' Symbol '):
match = re.fullmatch(r' Symbol (.+?)\tpage (.+)', line)
if match is None or block is None:
raise NativeInventoryUnsupported('分层报告图页身份不明确')
scopes[(block, match[1])] = match[2]
sheets.add((block, match[2]))
targets = []
for uid, row in physical.items():
sheets = {(row['source_schematic'], row['source_page']) for row in expanded}
counts = Counter(row.get('source_uid', uid) for uid, row in components.items())
targets = {}
for scope in expanded:
row = components[scope['graph_key']]
attributes = row['attributes']
if attributes.get('Part Number', '').casefold() not in requested:
if not _is_pcb(row) or attributes.get('Part Number', '').casefold() not in requested:
continue
if row.get('instance_path') or source_uids[uid] != 1 or references[row['refdes']] != 1:
raise NativeInventoryUnsupported('请求物料含子实例或非唯一 UID/位号,当前写入选择器不能安全定位')
if 'source_refdes' in row or row.get('ambiguous_attributes'):
raise NativeInventoryUnsupported('请求物料的位号或生效属性存在歧义')
sheet = scopes.get((root, uid))
if not sheet:
raise NativeInventoryUnsupported('请求物料缺少唯一根块图页')
targets.append(ComponentTarget(Path(project), root, sheet, uid, row['refdes'],
attributes['Part Number'], attributes.get('Value', ''), attributes.get('Part Label'),
attributes.get('Cell Name')))
if not scope.get('refdes'):
raise NativeInventoryUnsupported('Hierarchical physical component is missing its reference observation')
if 'Value' in scope['attributes'] and 'Value' not in attributes:
raise NativeInventoryUnsupported('Effective physical component is missing its Value observation')
if row.get('ambiguous_attributes'):
raise NativeInventoryUnsupported('Requested physical component has ambiguous effective attributes')
source_uid = row.get('source_uid', scope['source_uid'])
identity = source_uid if (_UID.fullmatch(source_uid) and counts[source_uid] == 1
and source_uid == scope['source_uid']) else '@REFDES|' + row['refdes']
target = ComponentTarget(Path(project), scope['source_schematic'], scope['source_page'],
identity, row['refdes'], attributes['Part Number'], attributes.get('Value', ''),
attributes.get('Part Label'), attributes.get('Cell Name'))
previous = targets.get(target.key)
if previous and (previous.old_part_number, previous.old_value, previous.actual_part_label,
previous.actual_cell) != (target.old_part_number, target.old_value,
target.actual_part_label, target.actual_cell):
raise NativeInventoryUnsupported('Conflicting effective attributes across physical symbol units')
targets[target.key] = target
matches = {sheet: set() for sheet in sheets}
for target in targets:
for target in targets.values():
matches[(target.schematic_name, target.sheet_number)].add(target.old_part_number.casefold())
return ProjectDiscovery(tuple(sorted(sheets)), {key: frozenset(value) for key, value in matches.items()},
tuple(sorted(targets, key=lambda target: target.key)), complete_sheet_manifest=False)
tuple(sorted(targets.values(), key=lambda target: target.key)), complete_sheet_manifest=False)
def read_ascii_discovery(project, report_path, old_part_numbers, *, expected_root=None,
@@ -107,10 +107,11 @@ def _parse_ascii(path, *, hierarchy_boundaries=(), require_connections=True):
symbol = pin = net = conn = None
for line in text.splitlines():
if line.startswith(' Symbol '):
match = re.fullmatch(r' Symbol (.+?)\tref des ([^\t]*)\tpage [^\t]*', line)
match = re.fullmatch(r' Symbol (.*?)\tref des ([^\t]*)\tpage ([^\t]*)', line)
if not match:
raise ValueError('Invalid effective symbol identity')
symbol = {'uid': match[1], 'refdes': match[2], 'attributes': {}, 'pins': {}, 'paths': []}
symbol = {'uid': match[1], 'refdes': match[2], 'source_page': match[3].strip(),
'attributes': {}, 'pins': {}, 'paths': []}
symbols.append(symbol)
pin = None
elif symbol is not None:
@@ -155,7 +156,7 @@ def _parse_ascii(path, *, hierarchy_boundaries=(), require_connections=True):
raise ValueError('Duplicate or empty net')
expected[net] = Counter()
elif line.startswith('\t\tConn '):
match = re.fullmatch(r'\t\tConn (.+?) pin (.+?) page .*', line)
match = re.fullmatch(r'\t\tConn (.*?) pin (.+?) page .*', line)
if net is None or not match:
raise ValueError('Invalid net endpoint')
conn = match.groups()
@@ -168,12 +169,15 @@ def _parse_ascii(path, *, hierarchy_boundaries=(), require_connections=True):
conn = None
if conn is not None:
raise ValueError('Endpoint missing path')
raw_counts = Counter((row['uid'], tuple(row['paths'])) for row in symbols)
endpoint_aliases = defaultdict(list)
for symbol in symbols:
uid = symbol.pop('uid')
paths = symbol.pop('paths')
if len(paths) != 1:
raise ValueError('Symbol instance path is missing or ambiguous')
source_uid = uid
symbol['source_uid'] = source_uid
uid = _instance_identity(uid, paths[0])
attrs = symbol['attributes']
if paths[0]:
@@ -181,8 +185,14 @@ def _parse_ascii(path, *, hierarchy_boundaries=(), require_connections=True):
symbol['refdes'] = attrs.get('Ref Designator', '')
elif attrs.get('Ref Designator') and attrs['Ref Designator'] != symbol['refdes']:
symbol['source_refdes'], symbol['refdes'] = symbol['refdes'], attrs['Ref Designator']
if _is_pcb(symbol) and (not re.fullmatch(r'\$\d+I\d+', source_uid) or not symbol['refdes'] or attrs.get('Ref Designator') != symbol['refdes']):
if _is_pcb(symbol) and (not symbol['refdes'] or attrs.get('Ref Designator') != symbol['refdes']):
raise ValueError(f'PCB component identity is ambiguous: {uid!r} / {symbol["refdes"]!r}')
raw_identity = uid
if _is_pcb(symbol) and (not re.fullmatch(r'\$\d+I\d+', source_uid)
or raw_counts[(source_uid, tuple(paths))] > 1):
slot = re.match(r'\S+\.(\d+) type ', symbol.get('symbol', ''))
uid = _instance_identity('@REFDES|' + symbol['refdes'] + ('|slot=' + slot[1] if slot else ''), paths[0])
endpoint_aliases[raw_identity].append(uid)
symbol['other_attributes'] = sorted(symbol.get('other_attributes', []))
if uid in components:
old = components[uid]
@@ -203,6 +213,16 @@ def _parse_ascii(path, *, hierarchy_boundaries=(), require_connections=True):
for net in row['nets']:
nets[net][(uid, pin)] += 1
for name, endpoints in expected.items():
normalized = Counter()
for (uid, pin), count in endpoints.items():
candidates = [key for key in endpoint_aliases.get(uid, [uid])
if key in components and pin in components[key]['pins']
and name in components[key]['pins'][pin]['nets']]
if len(set(candidates)) == 1:
uid = candidates[0]
normalized[(uid, pin)] += count
endpoints.clear()
endpoints.update(normalized)
for (uid, port), count in list(endpoints.items()):
if uid not in hierarchy_boundaries:
continue
@@ -316,18 +336,19 @@ def read_graph(ascii_path, keyin_path, *, hierarchy_path=None, root_block=None):
return graph
def _check_hierarchy_graph(path, graph, root_block):
"""Expand native child links independently; inventory UID selectors stay root-only."""
def _check_hierarchy_graph(path, graph, root_block, *, include_rows=False):
"""Bind expanded instances by reference/page/unit while retaining raw UID evidence."""
text = read_native_text(path)
_require_report(text, 'HIER BLOCKS')
blocks, roots, block = {}, [], None
blocks, roots, block, symbol = {}, [], None, None
for line in text.splitlines():
if line.startswith('Block '):
fields = line[6:].split('\t')
block = fields[0].rstrip()
if not block or block in blocks:
raise ValueError('Duplicate or missing native hierarchy block')
blocks[block] = {'children': {}, 'symbols': Counter()}
blocks[block] = {'children': {}, 'symbols': Counter(), 'rows': []}
symbol = None
if fields[1:] == ['Root']:
roots.append(block)
elif line.startswith('\tChild '):
@@ -336,13 +357,23 @@ def _check_hierarchy_graph(path, graph, root_block):
raise ValueError('Ambiguous native hierarchy child')
blocks[block]['children'][match[2]] = match[1]
elif line.startswith(' Symbol '):
match = re.fullmatch(r' Symbol (.+?)\tpage (.+)', line)
match = re.fullmatch(r' Symbol (.*?)\tpage (.+)', line)
if block is None or match is None:
raise ValueError('Ambiguous native hierarchy symbol')
blocks[block]['symbols'][match[1]] += 1
symbol = {'source_uid': match[1], 'source_page': match[2], 'attributes': {}}
blocks[block]['rows'].append(symbol)
elif symbol is not None and line.startswith('\tProp Ref Designator '):
symbol['refdes'] = line[len('\tProp Ref Designator '):]
elif symbol is not None and line.startswith('\tProp Part Number '):
symbol['attributes']['Part Number'] = line[len('\tProp Part Number '):]
elif symbol is not None and (line == '\tProp Value' or line.startswith('\tProp Value ')):
symbol['attributes']['Value'] = line[len('\tProp Value '):]
elif symbol is not None and line.startswith('\tSymbol name '):
symbol['symbol'] = line[len('\tSymbol name '):]
if roots != [root_block]:
raise ValueError('Native hierarchy root differs from captured project')
identities, boundaries = Counter(), set()
identities, boundaries, expanded = Counter(), set(), []
pending = [(root_block, '', set())]
while pending:
@@ -352,6 +383,10 @@ def _check_hierarchy_graph(path, graph, root_block):
row = blocks[name]
identities.update({_instance_identity(uid, path): count for uid, count in row['symbols'].items()
if uid not in row['children']})
for item in row['rows']:
if item['source_uid'] in row['children'] and not _is_pcb(item):
continue
expanded.append(dict(item, instance_path=path, source_schematic=name))
for uid, child in row['children'].items():
if uid not in row['symbols']:
raise ValueError('Native hierarchy child symbol missing')
@@ -359,10 +394,39 @@ def _check_hierarchy_graph(path, graph, root_block):
boundaries.add(identity)
pending.append((child, identity, ancestors | {name}))
if graph is not None:
effective = Counter({uid: len(row.get('annotation_instances', [row])) for uid, row in graph['components'].items()})
if identities != effective:
remaining = [(uid, item) for uid, row in graph['components'].items()
for item in row.get('annotation_instances', [row])]
indices, consumed = defaultdict(list), set()
for index, (uid, row) in enumerate(remaining):
instance = row.get('instance_path', uid.rsplit('/', 1)[0] if '/' in uid else '')
ref = row.get('source_refdes', row.get('refdes', '')).strip().casefold()
indices[(instance, 'ref', ref)].append(index)
indices[(instance, 'uid', row.get('source_uid', uid.rsplit('/', 1)[-1]))].append(index)
for item in expanded:
ref = item.get('refdes', '').strip().casefold()
key = item['instance_path'], 'ref' if ref else 'uid', ref or item['source_uid']
candidates = [(index, remaining[index][0], remaining[index][1]) for index in indices[key]
if index not in consumed and (not remaining[index][1].get('source_page')
or remaining[index][1]['source_page'] == item['source_page'])]
if len(candidates) > 1 and item.get('symbol'):
unit = item['symbol'].split(' type ', 1)[0]
candidates = [candidate for candidate in candidates if candidate[2].get('symbol', '').split(' type ', 1)[0] == unit]
if len(candidates) > 1:
exact = [candidate for candidate in candidates if candidate[2].get('source_uid') == item['source_uid']]
if exact:
candidates = exact
# Identical nonphysical annotations are counted, not mistaken for devices.
if not candidates or (len(candidates) > 1 and ref):
raise ValueError('Hierarchical/effective instance coverage differs')
index, uid, row = candidates[0]
item['graph_key'] = uid
if row:
row['source_schematic'] = item['source_schematic']
row['source_page'] = row.get('source_page') or item['source_page']
consumed.add(index)
if len(consumed) != len(remaining):
raise ValueError('Hierarchical/effective instance coverage differs')
return identities, boundaries
return (identities, boundaries, expanded) if include_rows else (identities, boundaries)
def export_netlist(capture: dict, output: Path, *, stop_event=None) -> dict:
@@ -466,6 +530,80 @@ def _connections(graph):
return sorted(nets)
def reference_targets(graph, refdes, *, sheet='', schematic=''):
"""Return every native unit for a physical reference within the requested sheet."""
ref = refdes.strip().casefold()
candidates = [(uid, row) for uid, row in graph['components'].items()
if _is_pcb(row) and row.get('refdes', '').strip().casefold() == ref]
pages_known = any(row.get('source_page') for _, row in candidates)
blocks_known = any(row.get('source_schematic') for _, row in candidates)
return tuple(uid for uid, row in candidates
if (not sheet or not pages_known or row.get('source_page') == sheet)
and (not schematic or not blocks_known or row.get('source_schematic') == schematic))
def _reference_graph(graph):
"""Normalize physical endpoints without changing the captured source evidence."""
from copy import deepcopy
result = deepcopy(graph)
identities, components = {}, {}
groups = defaultdict(list)
for row in result['components'].values():
if _is_pcb(row):
groups[row.get('refdes', '').strip().casefold()].append(row)
for uid, row in result['components'].items():
if _is_pcb(row):
ref = row.get('refdes', '').strip().casefold()
if not ref:
raise ValueError('Physical component has no reference designator')
# Separate slots of the same physical device remain independently checked.
peers = groups[ref]
scopes = {(item.get('source_schematic', ''), item.get('source_page', '')) for item in peers}
scope = (row.get('source_schematic', ''), row.get('source_page', ''))
same_scope = sum((item.get('source_schematic', ''), item.get('source_page', '')) == scope for item in peers)
slot = re.match(r'\S+\.(\d+) type ', row.get('symbol', ''))
identity = '@REFDES|' + ref
if len(scopes) > 1:
identity += '|sheet=' + json.dumps(scope, ensure_ascii=False)
if same_scope > 1 and slot:
repeated_slot = sum(bool(re.match(r'\S+\.' + re.escape(slot[1]) + r' type ', item.get('symbol', '')))
for item in peers if (item.get('source_schematic', ''), item.get('source_page', '')) == scope) > 1
if repeated_slot:
numbers = [pin.get('number') for pin in row.get('pins', {}).values()]
if not numbers or any(not number for number in numbers):
raise ValueError('Repeated symbol slots lack a unique physical pin partition')
identity += '|pins=' + json.dumps(sorted(numbers), ensure_ascii=False)
else:
identity += '|slot=' + slot[1]
row.pop('source_uid', None)
row.pop('instance_path', None)
else:
identity = uid
if identity in components:
raise ValueError('Reference designator and symbol slot are ambiguous')
identities[uid] = identity
components[identity] = row
result['components'] = components
for net in result['nets']:
net['endpoints'] = [[identities.get(uid, uid), pin] for uid, pin in net['endpoints']]
return result, identities
def _reference_expectations(graph, aliases, values):
result = {}
for uid, value in (values or {}).items():
targets = (uid,) if uid in aliases else ()
if not targets and uid.startswith('@REFDES|'):
targets = reference_targets(graph, uid[len('@REFDES|'):])
scopes = {(graph['components'][key].get('source_schematic', ''),
graph['components'][key].get('source_page', '')) for key in targets}
if len(scopes) > 1:
raise ValueError('Expected reference requires a sheet-qualified native identity')
for target in targets or (uid,):
result[aliases.get(target, target)] = value
return result
def compare_netlists(before: dict, after: dict, *, expected_changes=None, expected_symbols=None) -> dict:
"""Compare complete nonempty exports, allowing only declared attribute/symbol changes.
@@ -476,7 +614,13 @@ def compare_netlists(before: dict, after: dict, *, expected_changes=None, expect
graph = result.get('graph', {})
if result.get('status') != 'success' or result.get('complete') is not True or not graph.get('components') or not graph.get('nets'):
return {'status': 'unavailable', 'verified': False, 'reason': 'Complete nonempty native graphs required'}
left, right = before['graph'], after['graph']
try:
left, aliases = _reference_graph(before['graph'])
right, _ = _reference_graph(after['graph'])
expected_changes = _reference_expectations(before['graph'], aliases, expected_changes)
expected_symbols = _reference_expectations(before['graph'], aliases, expected_symbols)
except (KeyError, TypeError, ValueError) as error:
return {'status': 'unavailable', 'verified': False, 'reason': str(error)}
differences = []
try:
connections_changed = _connections(left) != _connections(right)
@@ -76,7 +76,8 @@ def library_from_receipt(command, receipt):
binding = require_resumable_transaction(snapshot, report, metadata, records)
if (canonical_path(metadata[1]) != canonical_path(command['project_path'])
or metadata[2:4] != [command['old_part'], command['new_part']]
or metadata[4] != f"{command['schematic_name']}:{command['sheet_number']}"):
or metadata[4] != f"{command['schematic_name']}:{command['sheet_number']}"
or (metadata[9] if len(metadata) == 10 else '') != command.get('target_selectors', '')):
raise ValueError('原生库件证据属于不同执行项')
row = binding['library_row']
if row[1].casefold() != command['new_part'].casefold() or not row[4]:
@@ -95,7 +96,9 @@ def library_from_receipt(command, receipt):
raise ValueError('Value 回读证据的器件范围不完整')
for uid in binding['source_uids']:
identity = {'attributes': {'Part Number': row[1], 'Part Name': row[4], 'Part Label': row[5]},
'target_symbol': binding['target_symbol'], 'slot_count': binding['slot_count']}
'target_symbol': binding['target_symbol'], 'slot_count': binding['slot_count'],
'refdes': next(record[2] for record in records if record[1] == uid),
'schematic_name': command.get('schematic_name',''), 'sheet_number': command.get('sheet_number','')}
if uid in values:
value = values[uid]
if (value['actual_part_number'].casefold() != row[1].casefold()
@@ -180,30 +183,82 @@ def _rule_rows(child, task_id, plan, before):
if not identities:
raise ValueError('规则替换缺少绑定实际 sheet/UID 的原生事务证据')
if selector is not None:
original = before['graph']['components'].get(selector[0], {})
if (set(identities) != {selector[0]} or original.get('refdes') != selector[1]
or original.get('attributes', {}).get('Value', '') != selector[2]):
keys = native_netlist.reference_targets(before['graph'], selector[1],
sheet=command.get('sheet_number',''),schematic=command.get('schematic_name',''))
if ((set(identities) != {selector[0]} and not all(
identity.get('refdes','').strip().casefold() == selector[1].strip().casefold()
for identity in identities.values())) or not keys or any(
before['graph']['components'][key].get('attributes', {}).get('Value', '') != selector[2]
for key in keys)):
raise ValueError('规则位号选择器与替换前器件及原生 UID 范围不一致')
scopes.append({key: command.get(key, '') for key in
('project_path', 'schematic_name', 'sheet_number', 'old_part', 'new_part')}
| {'source_uids': sorted(identities)})
for uid in identities:
component = before['graph']['components'].get(uid)
if component is None or component['attributes'].get('Part Number', '').casefold() != command['old_part'].casefold():
identity = identities[uid]
keys = _reference_keys(before['graph'], dict(identity, uid=uid,
sheet_number=command.get('sheet_number',''), schematic_name=command.get('schematic_name','')))
if not keys or any(before['graph']['components'][key]['attributes'].get('Part Number', '').casefold()
!= command['old_part'].casefold() for key in keys):
raise ValueError('原生 UID 无法唯一对应替换前的原料号;不能按同料号猜测图页')
row = {'enabled': True, 'target': {'uid': uid, 'old_value': component['attributes'].get('Value', '')},
for key in keys:
component = before['graph']['components'][key]
row = {'enabled': True, 'target': {'uid': key, 'refdes': component['refdes'],
'schematic_name':command.get('schematic_name',''),'sheet_number':command.get('sheet_number',''),
'old_value': component['attributes'].get('Value', '')},
'new_part_number': command['new_part'], 'value_mode': command['value_mode'],
'custom_value_enabled': command['value_mode'] == 'custom',
'custom_value': command.get('custom_value', ''), 'sync_nc': command.get('sync_nc', '1') == '1',
'target_cell': command.get('target_cell', '')}
if uid in rows and rows[uid] != row:
raise ValueError('同一 UID 对应多个不同规则结果,不能独立核验')
rows[uid] = row
if key in rows and rows[key] != row:
raise ValueError('同一位号对应多个不同规则结果,不能独立核验')
rows[key] = row
if not rows:
raise ValueError('已保存的规则替换缺少可核验的原生 UID 事务')
return list(rows.values()), scopes
def _reference_keys(graph, identity):
ref = identity.get('refdes', '').strip().casefold()
keys = native_netlist.reference_targets(graph, ref, sheet=identity.get('sheet_number',''),
schematic=identity.get('schematic_name','')) if ref else ()
uid = identity.get('uid')
if not keys and uid in graph['components']:
component = graph['components'][uid]
constraints = ((ref, component.get('refdes','').strip().casefold()),
(identity.get('sheet_number',''), component.get('source_page','')),
(identity.get('schematic_name',''), component.get('source_schematic','')))
if not any(expected and actual and expected != actual for expected, actual in constraints):
keys = (uid,)
return keys
def _reference_plan(plan, before, library):
from copy import deepcopy
result, rebound = deepcopy(plan), {}
components = before['graph']['components']
rows = []
for row in result['rows']:
target = row['target']
keys = _reference_keys(before['graph'], target)
if not keys and row['enabled']:
raise ValueError('独立网表无法定位计划中的位号')
for key in keys:
item = deepcopy(row)
item['target']['uid'] = key
rows.append(item)
if target['uid'] in library:
rebound[key] = library[target['uid']]
for uid, identity in library.items():
keys = _reference_keys(before['graph'], dict(identity, uid=uid))
if not keys:
raise ValueError('独立网表无法定位回执中的位号或图页')
for key in keys:
rebound[key] = identity
result['rows'] = rows
return result, rebound
class NetlistVerification:
def __init__(self, root, run_id, stop_event):
self.root, self.run_id, self.stop = Path(root), run_id, stop_event
@@ -398,6 +453,7 @@ class NetlistVerification:
rows, scopes = _rule_rows(record['child'], task_id, plan, before)
plan = dict(plan, rows=rows)
result['rule_scopes'] = scopes
plan, library = _reference_plan(plan, before, library)
expected = self._expected(plan, after, library)
symbols = {uid: {key: item[key] for key in ('target_symbol', 'slot_count')}
for uid, item in library.items() if uid in expected}
@@ -0,0 +1,64 @@
"""Small append-only call timeline; diagnostics never replace an execution error."""
from datetime import datetime, timezone
from contextlib import contextmanager
import json
import os
from .path_access import internal_path
def record_operation(root, operation, event, *, item_id='', error=''):
if event not in {'enter', 'return', 'error'}:
raise ValueError('operation event must be enter, return or error')
try:
record = {'timestamp_utc': datetime.now(timezone.utc).isoformat(),
'operation': operation, 'event': event, 'item_id': item_id}
if error:
record['error'] = str(error)
line = json.dumps(record, ensure_ascii=False) + '\n'
path = internal_path(root) / 'operation-events.jsonl'
path.parent.mkdir(parents=True, exist_ok=True)
with path.open('a', encoding='utf-8') as stream:
stream.write(line)
stream.flush()
os.fsync(stream.fileno())
return True
except Exception:
return False
@contextmanager
def operation_span(root, operation, *, item_id=''):
"""Trace a real call without letting diagnostics change its outcome."""
def publish(event, error=''):
try:
record_operation(root, operation, event, item_id=item_id, error=error)
except Exception:
pass
publish('enter')
try:
yield
except BaseException as error:
publish('error', error)
raise
else:
publish('return')
def summarize_operations(records):
active, errors, last_item = {}, [], None
for record in records:
if not isinstance(record, dict) or not all(isinstance(record.get(name, ''), str)
for name in ('operation', 'item_id', 'event')):
continue
key = record.get('operation'), record.get('item_id', '')
if record.get('event') == 'enter':
active[key] = record
elif record.get('event') in {'return', 'error'}:
active.pop(key, None)
if record['event'] == 'error':
errors.append(record)
elif record.get('operation') == 'item' and record.get('item_id'):
last_item = record['item_id']
return {'last_successful_item': last_item, 'incomplete_operations': list(active.values()),
'errors': errors}
@@ -0,0 +1,158 @@
"""Keep uncertain engineering sources quarantined without stopping unrelated work."""
from pathlib import Path
import json
import re
from .path_access import canonical_path, internal_path
def write_sources(project):
project = canonical_path(project)
sources = {str(project).casefold()}
try:
from .native_inventory_protocol import read_native_text
text = read_native_text(internal_path(project))
match = re.search(r'^\s*KEY\s+iCDBDir\s+"([^"\r\n]+)"', text, re.MULTILINE | re.IGNORECASE)
if match:
directory = Path(match[1])
if not directory.is_absolute():
directory = project.parent / directory
sources.add(str(canonical_path(directory / 'icdb.dat')).casefold())
except (OSError, UnicodeError, ValueError):
pass
return sources
def record_task_outcome(child, task_id, state, message):
store = getattr(child, '_store', None)
if store is None:
return
from .models import TaskStatus
target = {'blocked': TaskStatus.BLOCKED, 'failed': TaskStatus.FAILED, 'paused': TaskStatus.PAUSED}.get(state)
if target is None:
return # Successful completion is committed with its actual execution receipts.
with store.transaction():
task = store.get_task(task_id)
if task.status not in {TaskStatus.COMPLETED, TaskStatus.DELETED}:
store.set_task_status(task_id, target, summary=message)
def isolate_project(runtime, child, task_id, error):
release = getattr(runtime, 'release_failed_project', None)
if release is None:
return False
preparer = getattr(child, '_quick_run_preparer', None)
root = preparer.data_root / 'quick-run' if preparer is not None else runtime.root / 'tasks' / task_id / 'quick-run'
released = release(error, root)
return released
def dependency_path(root, task_id):
return canonical_path(root) / 'dependency-blocks' / (task_id + '.json')
def dependency_block(root, task_id):
from .errors import TaskCenterError
try:
value = json.loads(internal_path(dependency_path(root, task_id)).read_text(encoding='utf-8'))
if (not isinstance(value, dict) or value.get('version') != 1 or value.get('task_id') != task_id or value.get('undispatched') is not True
or value.get('state') not in {'blocked', 'readmission_prepared', 'readmitted'}
or not isinstance(value.get('dependency_task_id'), str)
or Path(value['dependency_task_id']).name != value['dependency_task_id']
or not isinstance(value.get('sources'), list) or not value['sources']
or not all(isinstance(source, str) and Path(source).is_absolute() for source in value['sources'])
or not isinstance(value.get('plan_hash'), str) or not re.fullmatch('[0-9a-f]{64}', value['plan_hash'])
or not isinstance(value.get('queue_id'), str) or not re.fullmatch('[0-9a-f]{32}', value['queue_id'])
or not isinstance(value.get('project_path'), str) or not Path(value['project_path']).is_absolute()):
raise ValueError('invalid dependency block')
return value
except FileNotFoundError:
return None
except (OSError, ValueError, TypeError) as error:
raise TaskCenterError('quick_run_dependency', '共享数据源隔离记录无法核验,请保留证据。') from error
def block_dependency(root, child, task_id, dependency_task_id, sources, plan_hash, queue_id):
from .batch_resources import write_record
projects = child.projects(task_id)
write_record(dependency_path(root, task_id), {'version': 1, 'task_id': task_id,
'dependency_task_id': dependency_task_id, 'sources': sorted(sources), 'plan_hash': plan_hash,
'project_path': str(canonical_path(projects[0].path)).casefold(),
'queue_id': queue_id, 'state': 'blocked', 'undispatched': True})
record_task_outcome(child, task_id, 'blocked', f'共享数据源依赖 {dependency_task_id} 尚待核验;本任务未派发替换。')
def _validate_dependency(service, task_id, block):
from .batch_resources import resource_blockers
from .errors import TaskCenterError
from .models import TaskStatus
from .task_queue import _plan_hash
child = service.for_project_task(task_id)
dependency = service.for_project_task(block['dependency_task_id'])
projects = child.projects(task_id)
dependency_projects = dependency.projects(block['dependency_task_id'])
queue_path = next((path for directory in ('task-queues', 'workspace-queues')
if internal_path(path := canonical_path(service._logging.root) / directory / (block['queue_id'] + '.json')).is_file()), None)
try:
queue = json.loads(internal_path(queue_path).read_text(encoding='utf-8')) if queue_path else {}
queue_bound = (queue['plan_hashes'][task_id] == block['plan_hash']
and queue['dependency_tasks'][task_id] == block['dependency_task_id']
and queue['entries'][task_id]['state'] == 'blocked')
except (OSError, ValueError, TypeError, KeyError):
queue_bound = False
store = getattr(child, '_store', None)
with store._lock:
dispatched = store._connection.execute('''SELECT 1 FROM execution_operations
WHERE task_id=? AND (attempt_id IS NOT NULL OR generation>0 OR state NOT IN ('pending')) LIMIT 1''',
(task_id,)).fetchone()
evidence = child._quick_run_preparer.data_root / 'quick-run'
absent_attempt = not dispatched and not any(internal_path(evidence).glob('snapshot-*'))
if (len(projects) != 1 or len(dependency_projects) != 1
or not queue_bound or not absent_attempt
or str(canonical_path(projects[0].path)).casefold() != block['project_path']
or _plan_hash(child, task_id) != block['plan_hash']
or not set(block['sources']) <= write_sources(projects[0].path) & write_sources(dependency_projects[0].path)
or child.has_interrupted_quick_run(task_id)
or dependency.has_interrupted_quick_run(block['dependency_task_id'])
or dependency.get_task(block['dependency_task_id']).status not in {TaskStatus.READY, TaskStatus.COMPLETED}
or resource_blockers(service.pending_batch_resources())):
raise TaskCenterError('quick_run_dependency', '原依赖工程尚未核验、写入资源未退出或冻结计划已变化,不能解除隔离。')
def recover_dependency(service, task_id, *, confirmed):
from .batch_resources import write_record
from .errors import TaskCenterError
from .models import TaskStatus
from .quick_run_protocol import QuickRunStepResult
root = service._logging.root
block = dependency_block(root, task_id)
if not confirmed:
raise TaskCenterError('quick_run_dependency', '请确认核验共享数据源依赖后解除未派发任务的隔离。')
_validate_dependency(service, task_id, block)
path = dependency_path(root, task_id)
write_record(path, dict(block, state='readmission_prepared'))
with service._store.transaction():
task = service._store.get_task(task_id)
if task.status is TaskStatus.BLOCKED:
service._store.set_task_status(task_id, TaskStatus.READY, summary='共享数据源依赖已核验;原计划未派发,可重新确认执行。')
elif task.status not in {TaskStatus.READY, TaskStatus.COMPLETED}:
raise TaskCenterError('quick_run_dependency', '当前任务状态已变化,请重新读取隔离记录。')
write_record(path, dict(block, state='readmitted'))
child = service.for_project_task(task_id)
progress = child.quick_run_progress(task_id)
handoff = child._quick_run_preparer.data_root / 'quick-run'
return QuickRunStepResult(handoff / 'QuickPartReplace.vbs', handoff / 'command.tsv', handoff / 'status.tsv', path,
'pending', '共享数据源依赖已核验;本任务从未派发替换,已恢复原待执行计划。',
progress.next_project_path, progress.next_old_part_number, progress.next_new_part_number,
progress.next_schematic_name, progress.next_sheet_number, progress)
def dependency_readmitted(service, task_id, plan_hash):
from .models import TaskStatus
block = dependency_block(service._logging.root, task_id)
if not block or block['state'] not in {'readmission_prepared', 'readmitted'} or block['plan_hash'] != plan_hash:
return False
if service.get_task(task_id).status not in {TaskStatus.READY, TaskStatus.COMPLETED}:
return False
_validate_dependency(service, task_id, block)
return True
@@ -2,12 +2,14 @@
from __future__ import annotations
from pathlib import Path
import json
import subprocess
import time
from threading import Event, Thread
from uuid import uuid4
from .worker_monitor import WorkerMonitorError, run_monitored_process
from .operation_diagnostics import operation_span
def _write_control(path: Path, value: str):
@@ -28,7 +30,7 @@ def _write_control(path: Path, value: str):
class ProjectSessionWorker:
"""Only a saved-and-reopened checkpoint releases the caller for its next step.
"""Retain applied steps; engineering close performs the save/readback commit.
The VBS worker owns CAD. This driver never attaches to COM or terminates CAD.
A separate monitor thread retains the existing idle watchdog and worker job.
@@ -61,6 +63,10 @@ class ProjectSessionWorker:
self._done.set()
def run_step(self, command: tuple[str, ...], timeout: float):
with operation_span(self.root, 'native_step', item_id=command[-1]):
return self._run_step(command, timeout)
def _run_step(self, command: tuple[str, ...], timeout: float):
if self._closed:
raise WorkerMonitorError('project session is already closed')
if Path(command[3]).resolve() != self.root.resolve():
@@ -82,6 +88,7 @@ class ProjectSessionWorker:
if not self._waiting:
raise WorkerMonitorError('project session has no verified checkpoint')
self._waiting = False
_write_control(self.root / 'project-session.request', f'{self.token}\t{self.project}\t{job}')
_write_control(self.root / 'project-session-next.tsv', f'{self.token}\t{job}')
while True:
if self._done.is_set():
@@ -92,8 +99,8 @@ class ProjectSessionWorker:
raise self._error
if self._result.returncode == 0:
raise WorkerMonitorError('project session exited before authorized closing')
if self._result.returncode == 74:
pause = getattr(self._result, 'startup_pause', {})
if self._result.returncode in {74, 75}:
pause = getattr(self._result, 'startup_pause' if self._result.returncode == 74 else 'prewrite_pause', {})
if (pause.get('session_token') != self.token or pause.get('job_id') != job
or Path(pause.get('project_path', '')).resolve() != self.project.resolve()
or pause.get('worker_tree_exited') is not True):
@@ -110,13 +117,20 @@ class ProjectSessionWorker:
except (FileNotFoundError, PermissionError):
fields = ()
if len(fields) == 4 and fields[:2] == [self.token, job]:
if fields[2] not in {'saved_ok', 'no_match'} or fields[3] != 'session_verified':
if (fields[2], fields[3]) not in {('saved_ok', 'session_verified'), ('no_match', 'session_verified'),
('pending_save', 'session_staged'), ('pending_save', 'session_staged_no_match')}:
raise WorkerMonitorError('project session returned an invalid checkpoint')
self._waiting = True
return subprocess.CompletedProcess(command, 0, '', '')
self._done.wait(0.025)
def close(self):
if not self._closed and self._thread is not None and not self._done.is_set():
with operation_span(self.root, 'close_project', item_id=self.token):
return self._close()
return self._close()
def _close(self):
if self._closed:
return
try:
@@ -128,25 +142,33 @@ class ProjectSessionWorker:
from .stop_control import StopControl
interactive = isinstance(getattr(self.runtime, 'stop_event', None), StopControl)
close_grace = 60 if self.private_desktop else min(30, max(10, self._timeout))
if interactive:
# A failed close has its own deadline, separate from pause.
force_event = getattr(getattr(self.runtime, 'stop_event', None), 'force_event', None)
last_notice = time.monotonic()
deadline = last_notice + close_grace
while not self._done.wait(.1):
if force_event is not None and force_event.is_set():
self._abort.set()
break
if time.monotonic() >= deadline:
force_event = getattr(getattr(self.runtime, 'stop_event', None), 'force_event', None)
last_notice = time.monotonic()
deadline = last_notice + close_grace
last_progress = (-1, -1)
while not self._done.wait(.1):
if force_event is not None and force_event.is_set():
self._abort.set()
break
try:
native = json.loads((self.root / 'worker-progress.json').read_text(encoding='utf-8'))
progress = (native['sequence'], native.get('native_progress_frames', 0))
if all(type(value) is int for value in progress) and progress > last_progress:
last_progress = progress
deadline = time.monotonic() + close_grace
except (OSError, ValueError, KeyError, TypeError):
pass
if time.monotonic() >= deadline:
if interactive:
self.runtime.stop_event.request_failure_cleanup(self.root,
'project session did not acknowledge closing within its grace period')
self._abort.set()
break
if time.monotonic() - last_notice >= 15:
import logging
logging.getLogger(__name__).warning(
'project_session_awaiting_close token=%s', self.token)
last_notice = time.monotonic()
self._abort.set()
break
if time.monotonic() - last_notice >= 15:
import logging
logging.getLogger(__name__).warning(
'project_session_awaiting_close token=%s', self.token)
last_notice = time.monotonic()
if not self._done.wait(close_grace):
self._abort.set()
if not self._done.wait(30):
@@ -174,6 +196,7 @@ class ProjectSessionScope:
self.worker = None
self.private_desktop = private_desktop
self.runtime = runtime
self.on_commit = None
def __enter__(self):
return self
@@ -189,13 +212,25 @@ class ProjectSessionScope:
def prepare(self, project):
if self.worker is not None and (self.worker.project.resolve() != Path(project).resolve()
or self.worker._done.is_set()):
self.worker.close()
self.worker = None
self.close()
def close(self):
if self.worker is None:return
worker = self.worker
worker.close()
if self.on_commit is not None:
self.on_commit(worker)
self.on_commit = None
self.worker = None
def __exit__(self, error_type, error, traceback):
if self.worker is not None:
try:
self.worker.close()
except Exception:
self.close()
except Exception as cleanup_error:
self.worker._abort.set()
if error is not None:
if hasattr(error, 'add_note'):
error.add_note('Project session cleanup: ' + str(cleanup_error))
return False
raise
@@ -376,6 +376,7 @@ class QuickRunPreparer:
from .handoff_guard import assert_handoff_intact
assert_handoff_intact(root, task_id)
try:
self._commit_staged_project(task_id)
self._bind_journal_handoff(task_id, QuickRunResult(root/'QuickPartReplace.vbs',
root/'command.tsv', root/'status.tsv', root/'quick-part-replace.log', self.vbs_root_compatible))
except TaskCenterError:
@@ -387,6 +388,194 @@ class QuickRunPreparer:
return capture_result(self._data_root,root,row,outcome,
strict=self._process_runner is _run_process,persist=persist)
def _commit_staged_project(self, task_id, *, session_token=None, bind_source=False):
journal = getattr(self, 'journal', None)
if journal is None:return
from .execution_proof import capture_project_commit
from .operation_diagnostics import record_operation
for commit in journal.pending_project_commits(task_id):
if session_token is not None and commit['session_token'] != session_token:continue
root = Path(commit['evidence_root'])
record_operation(root, 'readback_proof', 'enter', item_id=commit['session_token'])
try:
commit_proof = capture_project_commit(self._data_root, root, commit)
if commit.get('proof_json'):
commit_proof = json.loads(commit['proof_json'])
if commit_proof.get('saved_sources'):
from .execution_proof import validate_project_commit_source
if (not validate_project_commit_source(commit_proof, root)
or capture_project_commit(self._data_root, root, commit)['files'][0]['sha256'] != commit_proof['files'][0]['sha256']):
raise ValueError('saved project proof changed before receipt commit')
elif bind_source:
from .execution_proof import bind_project_commit_source
commit_proof = bind_project_commit_source(commit_proof, self._data_root, root, commit['project_path'])
commit_proof['operation_attempts'] = journal.project_commit_attempts(task_id, commit['session_token'])
proofs = {}
for receipt in journal.staged_project_rows(task_id, commit['session_token']):
row = receipt['row']
outcome = (commit_proof['outcomes'][row['job_id']], 'project_committed', '工程保存回读校验通过。')
proof = self.result_proof(root, row, outcome)
if outcome[0] == 'saved_ok':
from .value_readback import capture_value_readback
proof['value_readback'] = capture_value_readback(root, row, outcome, project_commit=commit_proof)
proofs[row['job_id']] = proof
if bind_source and not commit.get('proof_json'):
commit_proof['readback_files'] = {Path(item['path']).name: item['sha256']
for proof in proofs.values() for item in proof.get('files', ())}
for job, outcome in commit_proof['outcomes'].items():
if outcome == 'saved_ok':
value_path = _io_path(root / f'value-readback-{job}.tsv')
if value_path.is_file():commit_proof['readback_files'][value_path.name] = hashlib.sha256(value_path.read_bytes()).hexdigest()
journal.bind_project_commit_proof(task_id, commit['session_token'], commit_proof)
journal.commit_project_results(task_id, commit['session_token'], proofs, commit_proof)
record_operation(root, 'readback_proof', 'return', item_id=commit['session_token'])
for job in commit_proof['jobs']:
record_operation(root, 'item', 'return', item_id=job)
except Exception as error:
record_operation(root, 'readback_proof', 'error', item_id=commit['session_token'], error=str(error))
raise
def recover_project_commit(self, task_id, *, confirmed=False):
"""Reconcile an already closed/saved project without acquiring CAD."""
journal = getattr(self, 'journal', None)
if journal is None:return None
commits = journal.project_commit_records(task_id)
root = self._data_root / 'quick-run'
flag = _io_path(root / 'uncertain.flag')
audit = root / 'project-commit-recovery.json'
if not commits:return None
protection = flag.read_bytes() if flag.is_file() else None
allowed = {f'reason={reason}; manual reconciliation required\n'.encode() for reason in
('receipt_commit_failed', 'project_commit_unverified', 'uncommitted_receipt_requires_verification')}
interrupted = None
if protection is not None:
import re
match = re.fullmatch(rb'reason=supervisor_interrupted; batch=([0-9a-f]{32}); manual reconciliation required\r?\n', protection)
if match:interrupted = match[1].decode('ascii')
if protection is None:
try:
completed = json.loads(_io_path(audit).read_text(encoding='utf-8'))
if (completed.get('task_id') != task_id or completed.get('native_replacement_called') is not False
or completed.get('session_tokens') != [commit['session_token'] for commit in commits]):return None
except (OSError, ValueError, TypeError):return None
elif protection not in allowed and interrupted is None:return None
if not confirmed:
raise TaskCenterError('quick_run_recovery_confirmation', '工程提交回执核验需要明确确认。')
from .execution_proof import capture_project_commit, validate_project_commit_source
from .execution_journal import read_command_rows, command_identity, _hash
from .batch_resources import write_record
try:
rows = read_command_rows(root / 'command.tsv')
if not rows or any(row['task_id'] != task_id for row in rows):raise ValueError('foreign command plan')
current = journal._round(task_id)
if json.loads(current['plan_json']) != [_hash(command_identity(row)) for row in rows]:
raise ValueError('frozen command plan changed')
covered = set()
covered_attempts = []
for commit in commits:
if Path(commit['evidence_root']).resolve() != root.resolve():raise ValueError('foreign commit root')
proof = json.loads(commit['proof_json'] or '{}')
if proof and proof.get('operation_attempts') != journal.project_commit_attempts(task_id, commit['session_token']):
raise ValueError('original project attempts changed')
if not proof or interrupted is not None:
from .batch_resources import _valid_record
checkpoint = json.loads(_io_path(root / 'project-checkpoint.json').read_text(encoding='utf-8'))
owner_path = Path(checkpoint['resource_record'])
owner = json.loads(_io_path(owner_path).read_text(encoding='utf-8'))
proof = checkpoint['project_commit_source']
native = {key: checkpoint[key] for key in ('session_id', 'generation', 'attempt_id')}
if (not _valid_record(owner, owner_path) or checkpoint.get('worker_closed') is not True
or checkpoint.get('project_closed') is not True
or Path(checkpoint['evidence_root']).resolve() != root.resolve()
or _path_key(checkpoint['project_path']) != _path_key(commit['project_path'])
or owner['session_id'] != native['session_id'] or owner['generation'] != native['generation']
or owner['checkpoints'].get(str(root.resolve()).casefold()) != native['attempt_id']
or proof.get('native_attempt') != native or proof.get('session_token') != commit['session_token']
or proof.get('jobs') != json.loads(commit['jobs_json'])):
raise ValueError('native close checkpoint is not bound to this project commit')
if interrupted is not None:
supervisor = json.loads(_io_path(owner_path.parent / 'supervisor.json').read_text(encoding='utf-8'))
if (native['session_id'] != interrupted or supervisor.get('batch_id') != interrupted
or supervisor.get('state') != 'blocked' or supervisor.get('business_uncertain') is not True
or task_id not in supervisor.get('protected_task_ids', ())):
raise ValueError('supervisor interruption is not bound to the closed native worker')
proof = dict(proof, operation_attempts=journal.project_commit_attempts(task_id, commit['session_token']))
if (not validate_project_commit_source(proof, root)
or proof.get('operation_attempts') != journal.project_commit_attempts(task_id, commit['session_token'])):
raise ValueError('saved source or original attempts changed')
marker = capture_project_commit(self._data_root, root, commit)
if (marker['jobs'] != proof.get('jobs') or marker['outcomes'] != proof.get('outcomes')
or marker['files'][0]['sha256'] != proof['files'][0]['sha256']):
raise ValueError('native project marker changed')
covered.update(proof['jobs'])
covered_attempts.extend(proof['operation_attempts'].values())
for row in rows:
if row['job_id'] not in proof['jobs']:continue
outcome = (proof['outcomes'][row['job_id']], 'project_committed', '工程保存回读校验通过。')
result = self.result_proof(root, row, outcome)
if proof.get('native_attempt') and outcome[0] == 'saved_ok' and any(
result.get('native_attempt', {}).get(key) != value for key, value in proof['native_attempt'].items()):
raise ValueError('native result differs from the exited project worker')
if any(proof.get('readback_files', {}).get(Path(item['path']).name) != item['sha256'] for item in result.get('files', ())):
raise ValueError('saved native readback evidence changed')
if outcome[0] == 'saved_ok':
from .value_readback import capture_value_readback
if capture_value_readback(root, row, outcome, project_commit=proof).get('value_readback_status') != 'verified':
raise ValueError('saved target Value readback is incomplete')
if any(Path(name).name != name or hashlib.sha256(_io_path(root / name).read_bytes()).hexdigest() != digest
for name, digest in proof.get('readback_files', {}).items()):
raise ValueError('native/readback files changed since project closure')
if not commit.get('proof_json'):
journal.bind_project_commit_proof(task_id, commit['session_token'], proof)
commit['proof_json'] = json.dumps(proof)
if (not set(journal.unresolved_jobs(task_id, rows)) <= covered
or any(attempt not in covered_attempts for attempt in journal.project_commit_recovery_attempts(task_id))):
raise ValueError('another native attempt is unresolved')
# SQL/history may already have committed before a publication failure.
self._commit_staged_project(task_id)
for commit in commits:
proof = json.loads(commit['proof_json'])
if not validate_project_commit_source(proof, root):raise ValueError('saved source changed during recovery')
handoff = QuickRunResult(root/'QuickPartReplace.vbs', root/'command.tsv', root/'status.tsv', root/'quick-part-replace.log', self.vbs_root_compatible)
self._bind_journal_handoff(task_id, handoff)
if protection is not None:
if flag.read_bytes() != protection:raise ValueError('protection reason changed')
write_record(audit, {'task_id': task_id, 'session_tokens': [commit['session_token'] for commit in commits],
'native_replacement_called': False, 'original_protection_sha256': hashlib.sha256(protection).hexdigest()})
flag.replace(root / 'project-commit-resolved-uncertain.flag')
details = _read_status_details(root / 'status.tsv')
projects = tuple(project for project in journal.store.list_projects(task_id) if project.selected and project.available)
progress = _progress_from_queue(projects, _read_commands(root/'command.tsv'), details, _read_command_sheet_targets(root/'command.tsv'))
return QuickRunStepResult(handoff.script_path, handoff.command_path, handoff.status_path, audit,
'saved_ok', '完整工程保存回读证明已恢复原回执;未启动 CAD 或重复替换。', progress.next_project_path,
progress.next_old_part_number, progress.next_new_part_number, progress.next_schematic_name, progress.next_sheet_number, progress)
except (OSError, ValueError, KeyError, TypeError) as error:
raise TaskCenterError('quick_run_uncertain', '工程提交证明、保存源或原尝试无法核验,保护记录保持。') from error
def complete_project_session(self, task_id, scope, session, projects):
"""Close the retained owner before exposing saved progress to the queue."""
from .operation_diagnostics import record_operation
worker = scope.worker
if worker is not None:
record_operation(worker.root, 'save', 'enter', item_id=worker.token)
try:
scope.close()
record_operation(worker.root, 'save', 'return', item_id=worker.token)
except Exception as error:
record_operation(worker.root, 'save', 'error', item_id=worker.token, error=str(error))
_write_uncertain_flag(worker.root / 'uncertain.flag', 'project_commit_unverified')
raise
if session is not None:
appended, offset, identity, stamp = _read_appended_status_details(session)
session.statuses += appended
session.status_offset, session.status_identity, session.status_stamp = offset, identity, stamp
session.latest = {job: (state, code) for job, state, code, _ in session.statuses}
session.next_index = _next_unfinished_index(session.commands, session.latest)
session.progress = _progress_from_queue(tuple(project for project in projects if project.selected and project.available),
session.commands, session.statuses, session.sheets)
session.completed_project_keys = {_path_key(path) for path in session.progress.completed_project_paths}
return session.progress
def write_expected_baselines(self, task_id, root, project_path):
journal=getattr(self,'journal',None)
if journal is None:return
@@ -403,7 +592,8 @@ class QuickRunPreparer:
from .worker_monitor import WorkerMonitorError
try:
with ProjectSessionScope(self._process_runner,
enabled=enabled and self._process_runner is _run_process, private_desktop=True, runtime=runtime) as scope:
enabled=enabled and self._process_runner is _run_process and getattr(self, 'journal', None) is not None,
private_desktop=True, runtime=runtime) as scope:
yield scope
except (WorkerMonitorError, subprocess.TimeoutExpired) as error:
_write_uncertain_flag(self._data_root / 'quick-run/uncertain.flag', 'project_session_cleanup')
@@ -636,6 +826,38 @@ class QuickRunPreparer:
_write_uncertain_flag(uncertain_path, "malformed_status_after_worker")
_append_log(handoff.log_path, "quick_run_uncertain malformed_status_after_worker")
raise TaskCenterError("quick_run_uncertain", "quick-run worker left an unreadable status; reconcile the project before retrying") from error
if completed.returncode == 75:
try:
proof = getattr(completed, 'prewrite_pause', {})
pause_token = proof.get('session_token', '')
marker = _io_path(handoff.command_path.parent / f'project-pause-{pause_token}.tsv')
fields = marker.read_text(encoding='utf-8-sig').strip().split('\t')
if (runtime is None or journal_attempt is None or project_session is None
or project_session.worker is None or pause_token != project_session.worker.token
or proof.get('phase') != 'before_native_dispatch'
or proof.get('session_id') != runtime.session_id or proof.get('generation') != runtime.generation
or proof.get('attempt_id') != runtime.record.get('active_attempt')
or proof.get('resource_record') != str(runtime.path)
or len(fields) != 4 or fields[0] != pause_token or fields[2:] != [row['job_id'], 'prewrite_closed']
or _path_key(_decode_vbs_field(fields[1])) != _path_key(row['project_path'])
or proof.get('request_sha256') != hashlib.sha256(
_io_path(handoff.command_path.with_name('project-session.request')).read_bytes()).hexdigest()
or proof.get('current_command_sha256') != hashlib.sha256(
_io_path(handoff.command_path.with_name('current-command.tsv')).read_bytes()).hexdigest()
or any(entry[0] == row['job_id'] and entry[1] in TERMINAL_STATES for entry in after)
or not ((stop_event is not None and stop_event.is_set())
or (runtime.stop_event is not None and runtime.stop_event.is_set()))):
raise ValueError('prewrite pause is not bound to the retained owner')
self._commit_staged_project(task_id, session_token=pause_token)
journal.pause_before_dispatch(journal_attempt, row, proof)
journal.finalize_startup_pause(task_id, handoff.command_path.parent)
_write_status_details(handoff.status_path, (*after,
(row['job_id'], 'pending', 'prewrite_paused', '工程安全关闭;当前项确认未替换,保留待执行。')))
except Exception as error:
_write_uncertain_flag(uncertain_path, 'unproven_prewrite_pause')
raise TaskCenterError('quick_run_uncertain', '暂停缺少完整的未写入与工程关闭证明;保留保护并核验。') from error
_append_log(handoff.log_path, 'quick_run_paused prewrite_closed')
raise TaskCenterError('operation_cancelled', '工程已安全关闭,已暂存前缀完成保存回读;当前项保留待执行。')
if completed.returncode == 74:
try:
proof = getattr(completed, 'startup_pause', {})
@@ -771,7 +993,8 @@ class QuickRunPreparer:
_append_log(handoff.log_path, "quick_run_uncertain native_failure_after_snapshot")
raise TaskCenterError("quick_run_uncertain", detail)
raise TaskCenterError("quick_run_execution", detail)
if state not in TERMINAL_STATES:
if state not in TERMINAL_STATES and not (state == 'pending_save' and project_session is not None
and project_session.enabled and code in {'session_staged', 'session_staged_no_match'}):
raise TaskCenterError("quick_run_execution", f"unexpected quick-run state: {state}")
if batch_mode and state == "saved_ok":
try:
@@ -804,6 +1027,22 @@ class QuickRunPreparer:
progress = _progress_from_queue(tuple(project for project in projects if project.selected and project.available),
commands, after, _read_command_sheet_targets(handoff.command_path))
try:
if state == 'pending_save':
proof = self.result_proof(handoff.command_path.parent, row, (state, code, message))
journal.stage_project_result(journal_attempt, row, session_token=project_session.worker.token,
root=handoff.command_path.parent, proof=proof)
project_session.on_commit = lambda worker: self._commit_staged_project(task_id, session_token=worker.token, bind_source=True)
if (next_command is None or _path_key(next_command[2]) != _path_key(row['project_path'])
or (stop_event is not None and stop_event.is_set())):
progress = self.complete_project_session(task_id, project_session, batch_session, projects)
state = 'no_match' if code == 'session_staged_no_match' else 'saved_ok'
message = '工程统一保存、关闭重开及回读校验通过。'
next_command = commands[batch_session.next_index] if batch_session is not None and batch_session.next_index < len(commands) else None
next_sheet = batch_session.sheets.get(next_command[1], (None, None)) if batch_session is not None and next_command is not None and batch_session.sheets is not None else (None, None)
return QuickRunStepResult(handoff.script_path, handoff.command_path, handoff.status_path, handoff.log_path,
state, message, Path(next_command[2]) if next_command is not None else None,
next_command[3] if next_command is not None else None, next_command[4] if next_command is not None else None,
next_sheet[0], next_sheet[1], progress)
verified_source_health = None
if (state in {'saved_ok', 'restored_ok', 'no_match'} and self._process_runner is _run_process
and journal.failure_health_references(task_id, row['project_path'])):
@@ -1196,9 +1435,21 @@ def _advance_batch_session(
current_index = session.next_index
current = session.commands[current_index]
job_id, state, code, _ = changed
if job_id != current[1] or state not in TERMINAL_STATES:
if job_id != current[1] or state not in TERMINAL_STATES | {'pending_save'}:
raise TaskCenterError("quick_run_queue_progress", "batch result does not complete the expected queue item")
session.latest[job_id] = (state, code)
if state == 'pending_save':
from dataclasses import replace
next_index = current_index + 1
while next_index < len(session.commands) and session.latest.get(session.commands[next_index][1], ('', ''))[0] in TERMINAL_STATES | {'pending_save'}:
next_index += 1
session.next_index = next_index
next_command = session.commands[next_index] if next_index < len(session.commands) else None
next_sheet = session.sheets.get(next_command[1], (None, None)) if session.sheets is not None and next_command is not None else (None, None)
session.progress = replace(session.progress, next_project_path=Path(next_command[2]) if next_command else None,
next_old_part_number=next_command[3] if next_command else None, next_new_part_number=next_command[4] if next_command else None,
next_schematic_name=next_sheet[0], next_sheet_number=next_sheet[1], queue_complete=False)
return
current_key = session.project_keys_by_index[current_index]
partial_failures = {
_path_key(path): (path, summary)
@@ -2074,10 +2325,12 @@ def _correlated_status_change(
job_id = appended[0][0]
prior_state = next((state for status_job_id, state, _, _ in reversed(before) if status_job_id == job_id), None)
transitions = {
None: {"awaiting_native_replace", "hard_failure", "batch_failed", "batch_saved_unverified", "no_match"},
None: {"awaiting_native_replace", "hard_failure", "batch_failed", "batch_saved_unverified", "no_match", "pending_save"},
"pending": {"awaiting_native_replace", "hard_failure", "batch_failed", "batch_saved_unverified", "no_match", "pending_save"},
"awaiting_native_replace": {
"restored_ok",
"saved_ok",
"pending_save",
"hard_failure",
"batch_failed",
"batch_saved_unverified",
@@ -1117,8 +1117,9 @@ class TaskCenterService:
summary='quick_run_result:failed | '+message)
if task.status in {TaskStatus.DRAFT,TaskStatus.READY,TaskStatus.PAUSED}:
self._store.set_task_status(task_id,TaskStatus.RUNNING,summary=summary)
self._store.set_task_status(task_id,TaskStatus.FAILED,summary=summary)
elif task.status is TaskStatus.RUNNING:
self._store.set_task_status(task_id,TaskStatus.RUNNING,summary=summary)
self._store.set_task_status(task_id,TaskStatus.FAILED,summary=summary)
HistoryRepository(self._store).append(task_id=task_id,task_name=task.name,
operation='工程执行收尾',state='failed',summary='本工程安全失败,剩余项暂缓;继续后续工程。',
details={'reason':message})
@@ -1147,7 +1148,15 @@ class TaskCenterService:
child._quick_run_preparer.data_root, started_after)
if getattr(self, '_supervisor_waiting', False):
from dataclasses import replace
value = replace(value, stage='awaiting_reconciliation', waiting_for_safe_completion=True)
from .execution_feedback import wait_reason_for_stage
state = getattr(self, '_supervisor_state', None) or {}
wait_reason = state.get('wait_reason') or value.wait_reason or wait_reason_for_stage(value.stage)
stopping_reason = state.get('stopping_reason') or value.stopping_reason
if not stopping_reason and getattr(self, '_supervisor_business_stalled', False):
stopping_reason = 'inactivity'
value = replace(value, waiting_for_safe_completion=True,
wait_reason=wait_reason, stopping_reason=stopping_reason,
primary_error=state.get('primary_error') or value.primary_error)
if read_control is not None:
read_control.checkpoint()
return value
@@ -1175,6 +1184,35 @@ class TaskCenterService:
return self._quick_run_preparer.has_interrupted(task_id)
def recover_quick_run(self, task_id: str, *, confirmed: bool = False, runtime=None, stop_event=None) -> QuickRunStepResult:
if self._store.get_task(task_id).status is TaskStatus.DELETED:
raise TaskCenterError("quick_run_recovery", "请先恢复已删除的任务记录,再核验其中断步骤。")
from .project_isolation import dependency_block, recover_dependency
dependency = dependency_block(self._logging.root, task_id)
with self._store._lock:
dispatched = self._store._connection.execute('''SELECT 1 FROM execution_operations
WHERE task_id=? AND (attempt_id IS NOT NULL OR generation>0) LIMIT 1''', (task_id,)).fetchone()
if dependency is not None and not (dependency['state'] in {'readmission_prepared', 'readmitted'} and dispatched):
if stop_event is not None and stop_event.is_set():
raise TaskCenterError('operation_cancelled', '共享数据源依赖核验已取消。')
from .task_queue import assert_task_queue_idle
assert_task_queue_idle(self._logging.root)
with _quick_run_lease(self._logging.root):
return self._operation('recover_quick_run', task_id,
lambda: recover_dependency(self, task_id, confirmed=confirmed))
commit_recovery = getattr(self._project_task_service(task_id)._quick_run_preparer, 'recover_project_commit', None)
if commit_recovery is not None:
from .batch_resources import guard_pending_resources
from .task_queue import assert_task_queue_idle
assert_task_queue_idle(self._logging.root)
with _quick_run_lease(self._logging.root):
guard_pending_resources(self._logging.root)
result = self._operation('recover_project_commit', task_id,
lambda: commit_recovery(task_id, confirmed=confirmed))
if result is not None:
if self._store.get_task(task_id).status in {TaskStatus.BLOCKED, TaskStatus.FAILED, TaskStatus.PAUSED}:
self._store.set_task_status(task_id, TaskStatus.READY, summary='原工程提交已通过持久化保存回读证据核验。')
self._synchronize_quick_run_progress(task_id)
return result
if runtime is None and self._supervised:
return self._supervise('recover_quick_run', {'task_id': task_id, 'confirmed': confirmed}, stop_event=stop_event)
if runtime is None:
@@ -1221,7 +1259,7 @@ class TaskCenterService:
self._quick_run_preparer._bind_journal_handoff(task_id,
QuickRunResult(result.script_path, result.command_path, result.status_path,
handoff_root/'quick-part-replace.log', True))
if task.status in {TaskStatus.PAUSED, TaskStatus.FAILED}:
if task.status in {TaskStatus.PAUSED, TaskStatus.FAILED, TaskStatus.BLOCKED}:
self._store.set_task_status(task_id, TaskStatus.READY, summary="interrupted quick-run reconciled")
from .recovery_commit import marker_path
marker_path(self._quick_run_preparer).unlink(missing_ok=True)
@@ -1409,8 +1447,8 @@ class TaskCenterService:
rules,
copy_part_number_to_value=task.copy_part_number_to_value,
)
if self._supervised:
return progress # Only the batch process persists live execution state.
if self._supervised or task.status is TaskStatus.BLOCKED:
return progress # BLOCKED classification changes only through explicit reconciliation.
by_path = {_path_key(project.path): project.project_id for project in projects}
project_by_id = {project.project_id: project for project in projects}
summary_by_id = {
@@ -1457,7 +1495,7 @@ class TaskCenterService:
) or (
progress.queue_complete
and (
(bool(failed_project_ids) and (task.status is not TaskStatus.RUNNING or task.summary != retry_summary))
(bool(failed_project_ids) and (task.status is not TaskStatus.FAILED or task.summary != retry_summary))
or (not failed_project_ids and task.status is not TaskStatus.COMPLETED)
)
)
@@ -80,7 +80,7 @@ def read_target_plan(store, task_id: str, path: Path) -> tuple[TargetPlanRow, ..
raise ValueError("invalid or formula plan row")
identity = tuple("" if value is None else str(value) for value in values[:8])
key = (str(Path(identity[0]).resolve(strict=False)).casefold(),
identity[2], identity[3], identity[4])
identity[2], identity[3], identity[5].strip().casefold())
original = expected.get(key)
if original is None or key in by_key:
raise ValueError("forged or duplicate target")
@@ -1,5 +1,6 @@
"""Separate ordinary stop intent from an explicitly confirmed forced interruption."""
from threading import Event
import time
import json
import os
@@ -23,6 +24,13 @@ class StopControl(Event):
self.resolution_error = ''
self.cleanup_mode = None
self.cleanup_reason = ''
self.stopping_reason = ''
self.wait_reason = ''
self.primary_error = ''
def set(self):
self.stopping_reason = self.stopping_reason or 'user_pause'
super().set()
def request_force(self):
self.set()
@@ -33,10 +41,64 @@ class StopControl(Event):
protect_interruption(root, MENTOR_RELEASE_REQUIRED + ': ' + str(error))
self.cleanup_mode = MENTOR_RELEASE_REQUIRED
self.cleanup_reason = self.cleanup_reason or str(error)
self.primary_error = self.primary_error or str(error)
self.stopping_reason = 'failure_cleanup'
self.set()
self.force_event.set()
class _ProjectForce:
def __init__(self, parent):
self.local = Event()
self.parent = getattr(parent, 'force_event', None)
def is_set(self):
return self.local.is_set() or self.parent is not None and self.parent.is_set()
def set(self):
self.local.set()
def clear(self):
self.local.clear()
if self.parent is not None:
self.parent.clear()
def wait(self, timeout=None):
deadline = None if timeout is None else time.monotonic() + timeout
while not self.is_set():
remaining = .1 if deadline is None else min(.1, deadline - time.monotonic())
if remaining <= 0:
return False
self.local.wait(remaining)
return True
class ProjectStopControl(StopControl):
"""An engineering failure stops its owner; user pause still stops the batch."""
def __init__(self, parent):
super().__init__()
self.parent = parent
self.force_event = _ProjectForce(parent)
def is_set(self):
return self.parent.is_set() or super().is_set()
def set(self):
if self.stopping_reason in {'failure_cleanup','inactivity'}:
Event.set(self)
else:
self.parent.set()
def request_failure_cleanup(self, root, error):
protect_interruption(root, MENTOR_RELEASE_REQUIRED + ': ' + str(error))
self.stopping_reason = 'failure_cleanup'
self.cleanup_mode = MENTOR_RELEASE_REQUIRED
self.cleanup_reason = self.cleanup_reason or str(error)
self.primary_error = self.primary_error or str(error)
Event.set(self)
self.force_event.set()
def protect_interruption(root, reason):
"""Create or re-sync a marker without overwriting prior protection evidence."""
root = internal_path(root)
@@ -69,9 +131,12 @@ def wait_for_resolution(control, root, error, *, resolved=None, release_on_failu
return
from .worker_monitor import WorkerMonitorError, _publish_progress
root = internal_path(root)
control.stopping_reason = 'failure_cleanup' if release_on_failure else (control.stopping_reason or 'inactivity')
control.wait_reason = 'reconciliation'
control.primary_error = control.primary_error or str(error)
control.set()
control.waiting_for_resolution = True
control.resolution_error = str(error)
control.resolution_error = control.resolution_error or str(error)
def protect():
protect_interruption(root, 'operation_interrupted: ' + str(error))
try:
@@ -91,7 +156,9 @@ def wait_for_resolution(control, root, error, *, resolved=None, release_on_failu
record = {}
record.update(stage='awaiting_reconciliation', waiting_for_safe_completion=True,
resolution_error=str(error), stop_requested=True, safe_stop=False,
cleanup_mode=control.cleanup_mode)
cleanup_mode=control.cleanup_mode, stopping_reason=control.stopping_reason,
wait_reason=control.wait_reason,
primary_error=record.get('primary_error') or control.primary_error)
_publish_progress(root / 'worker-progress.json', record)
while True:
if resolved is not None and resolved():
+13 -5
View File
@@ -40,7 +40,7 @@ _TASK_TRANSITIONS = {
TaskStatus.RUNNING: {TaskStatus.PAUSED, TaskStatus.COMPLETED, TaskStatus.FAILED},
TaskStatus.PAUSED: {TaskStatus.READY, TaskStatus.RUNNING, TaskStatus.FAILED, TaskStatus.DELETED},
TaskStatus.COMPLETED: {TaskStatus.DELETED},
TaskStatus.FAILED: {TaskStatus.PAUSED, TaskStatus.READY, TaskStatus.DELETED},
TaskStatus.FAILED: {TaskStatus.PAUSED, TaskStatus.READY, TaskStatus.RUNNING, TaskStatus.DELETED},
TaskStatus.DELETED: {
TaskStatus.DRAFT,
TaskStatus.READY,
@@ -49,6 +49,13 @@ _TASK_TRANSITIONS = {
TaskStatus.FAILED,
},
}
for _state in (TaskStatus.DRAFT, TaskStatus.READY, TaskStatus.RUNNING, TaskStatus.PAUSED, TaskStatus.FAILED):
_TASK_TRANSITIONS[_state].add(TaskStatus.BLOCKED)
_TASK_TRANSITIONS[TaskStatus.DRAFT].update({TaskStatus.PAUSED, TaskStatus.FAILED})
_TASK_TRANSITIONS[TaskStatus.READY].add(TaskStatus.PAUSED)
_TASK_TRANSITIONS[TaskStatus.FAILED].add(TaskStatus.FAILED)
_TASK_TRANSITIONS[TaskStatus.BLOCKED] = {TaskStatus.BLOCKED, TaskStatus.PAUSED, TaskStatus.READY, TaskStatus.DELETED}
_TASK_TRANSITIONS[TaskStatus.DELETED].add(TaskStatus.BLOCKED)
_PROJECT_TRANSITIONS = {
ProjectStatus.PENDING: {ProjectStatus.CAPTURING_BEFORE, ProjectStatus.FAILED, ProjectStatus.SKIPPED},
ProjectStatus.CAPTURING_BEFORE: {ProjectStatus.AWAITING_OPERATOR, ProjectStatus.FAILED},
@@ -467,7 +474,8 @@ class TaskStore:
"""Persist current-sheet quick-run progress independently of manual attempts."""
with self.transaction():
task = self.get_task(task_id)
allowed_task_states = {TaskStatus.DRAFT, TaskStatus.READY, TaskStatus.RUNNING, TaskStatus.PAUSED, TaskStatus.COMPLETED}
allowed_task_states = {TaskStatus.DRAFT, TaskStatus.READY, TaskStatus.RUNNING, TaskStatus.PAUSED,
TaskStatus.COMPLETED, TaskStatus.FAILED}
if task.status not in allowed_task_states:
raise TaskCenterError("quick_run_task_state", "task state does not allow quick-run progress")
for project_id in completed_project_ids:
@@ -512,7 +520,7 @@ class TaskStore:
"quick-run queue completed before project states were synchronized",
)
has_failures = any(project.status is ProjectStatus.FAILED for project in selected)
if task.status in {TaskStatus.DRAFT, TaskStatus.READY, TaskStatus.PAUSED}:
if task.status in {TaskStatus.DRAFT, TaskStatus.READY, TaskStatus.PAUSED, TaskStatus.FAILED}:
task = self.set_task_status(
task_id,
TaskStatus.RUNNING,
@@ -520,10 +528,10 @@ class TaskStore:
if has_failures else "current-sheet quick-run queue completed",
)
if has_failures:
if task.status is TaskStatus.RUNNING and task.summary != "current-sheet quick-run queue completed with retryable failures":
if task.status is TaskStatus.RUNNING:
task = self.set_task_status(
task_id,
TaskStatus.RUNNING,
TaskStatus.FAILED,
summary="current-sheet quick-run queue completed with retryable failures",
)
else:
@@ -20,7 +20,8 @@ class ProjectIdentityResolver:
return self._projects[spelling]
def target(self, target):
return (self.project(target.project_path), target.schematic_name, target.sheet_number, target.uid)
return (self.project(target.project_path), target.schematic_name, target.sheet_number,
target.refdes.strip().casefold())
@dataclass(frozen=True, slots=True)
@@ -36,6 +37,8 @@ class ComponentTarget:
actual_cell: str | None = None
def __post_init__(self) -> None:
if not self.uid and self.refdes:
object.__setattr__(self, 'uid', '@REFDES|' + self.refdes.strip())
if any(value is not None and not isinstance(value, str)
for value in (self.actual_part_label, self.actual_cell)):
raise TaskCenterError('target_inventory', 'effective Part Label and Cell must be text or unknown')
@@ -118,7 +121,7 @@ def compile_target_rules(rows: tuple[TargetPlanRow, ...], *, default_value_mode:
raise TaskCenterError("target_identity", "component identity is incomplete")
target_key = identities.target(target)
if target_key in seen:
raise TaskCenterError("target_overlap", "target UID appears more than once")
raise TaskCenterError("target_overlap", "target reference appears more than once")
seen.add(target_key)
if not row.enabled:
continue
@@ -23,7 +23,8 @@ def archive_completed_session(service):
def _archive(service):
store = service._store
pending = service.pending_batch_resources()
from .batch_resources import resource_blockers
pending = resource_blockers(service.pending_batch_resources())
owned = set()
unknown = False
for record in pending:
@@ -39,7 +40,7 @@ def _archive(service):
reason = None
if unknown or task.task_id in owned:
reason = '后台资源尚未确认释放;重新打开后使用“重新清理资源”。'
elif task.status is TaskStatus.RUNNING:
elif task.status in {TaskStatus.RUNNING, TaskStatus.BLOCKED}:
reason = '任务仍在执行状态;重新打开后暂停或核验恢复。'
elif any(project.status in {ProjectStatus.CAPTURING_BEFORE, ProjectStatus.AWAITING_OPERATOR,
ProjectStatus.CAPTURING_AFTER, ProjectStatus.VERIFYING}
@@ -12,6 +12,8 @@ from uuid import uuid4
from .errors import TaskCenterError
from .quick_run import _write_atomic_queue_text
from .project_isolation import (write_sources, record_task_outcome, isolate_project,
block_dependency, dependency_block, dependency_readmitted)
_QUEUE_LOCK = Lock()
@@ -212,12 +214,14 @@ def execute_task_queue(service, task_ids: tuple[str, ...], *, stop_event: Event,
with _queue_lease(root):
children = {task_id: service.for_project_task(task_id) for task_id in ids}
project_paths = []
sources = {}
for task_id, child in children.items():
task = child.get_task(task_id)
projects = child.projects(task_id)
if not getattr(task, 'primary_project_id', None) or len(projects) != 1:
raise TaskCenterError('project_binding', '顺序执行仅支持绑定单个工程的任务。')
project_paths.append(str(projects[0].path.resolve(strict=False)).casefold())
sources[task_id] = write_sources(projects[0].path)
if len(set(project_paths)) != len(ids):
raise TaskCenterError('task_queue_selection', '不能在同一队列重复处理同一个工程。')
runtime.bind_logs(service._logging, ids)
@@ -251,10 +255,38 @@ def execute_task_queue(service, task_ids: tuple[str, ...], *, stop_event: Event,
_save(path, state)
paused = False
current = None
from .stop_control import ProjectStopControl
parent_control = getattr(runtime, 'stop_event', None)
isolated_tasks = state.setdefault('isolated_tasks', {})
isolated_sources = state.setdefault('isolated_sources', {})
dependency_tasks = state.setdefault('dependency_tasks', {})
from .netlist_verification import NetlistVerification
checks = NetlistVerification(root, token, stop_event)
try:
for task_id in ids:
readmitted = False
if task_id in dependency_tasks:
try:
readmitted = dependency_readmitted(children[task_id], task_id, hashes[task_id])
except TaskCenterError:
pass
if readmitted:
isolated_tasks.pop(task_id, None)
dependency_tasks.pop(task_id)
state['entries'][task_id] = {'state': 'pending', 'message': '共享依赖已核验,恢复未派发原计划。'}
if task_id in isolated_tasks:
continue
conflict = next((isolated_sources[source] for source in sources[task_id]
if source in isolated_sources), None)
if conflict is not None and not readmitted and state['entries'][task_id]['state'] not in _TERMINAL:
message = f'与待核验工程 {conflict} 共享写入源,已隔离;未派发替换。'
state['entries'][task_id] = {'state': 'blocked', 'message': message}
isolated_tasks[task_id] = message
dependency_tasks[task_id] = conflict
block_dependency(root, children[task_id], task_id, conflict,
sources[task_id] & sources[conflict], hashes[task_id], token)
_save(path, state)
continue
if state['entries'][task_id]['state'] in _TERMINAL:
if state['entries'][task_id]['state'] == 'completed':
checks.before_task(children[task_id], task_id, existing_only=True)
@@ -264,6 +296,10 @@ def execute_task_queue(service, task_ids: tuple[str, ...], *, stop_event: Event,
break
current = task_id
child = children[task_id]
project_control = ProjectStopControl(stop_event)
runtime.stop_event = project_control
if getattr(runtime, 'cad', None) is not None:
runtime.cad.stop_control = project_control
service._task_queue_feedback = (token, task_id, ids.index(task_id) + 1, len(ids))
resolving_bom = False
execution_dispatched = False
@@ -277,7 +313,7 @@ def execute_task_queue(service, task_ids: tuple[str, ...], *, stop_event: Event,
if _plan_hash(child, task_id) != hashes[task_id]:
raise TaskCenterError('assistant_stale', '待执行 BOM 已变化,请重新确认。')
resolving_bom = True
resolved = child.resolve_pending_bom(task_id, stop_event=stop_event)
resolved = child.resolve_pending_bom(task_id, stop_event=project_control)
resolving_bom = False
if not any(row.enabled for row in resolved):
state['entries'][task_id] = {'state': 'skipped',
@@ -304,7 +340,7 @@ def execute_task_queue(service, task_ids: tuple[str, ...], *, stop_event: Event,
for parameter in parameters) else {})
checks.before_task(child, task_id)
execution_dispatched = True
result = execute(task_id, expected_item=expected, stop_event=stop_event,
result = execute(task_id, expected_item=expected, stop_event=project_control,
**runtime_options)
if child.has_interrupted_quick_run(task_id) or (
result.last_result is not None and result.last_result.state == 'batch_saved_unverified'
@@ -340,10 +376,31 @@ def execute_task_queue(service, task_ids: tuple[str, ...], *, stop_event: Event,
entry = {'state': 'failed' if safe else 'paused' if error.code == 'operation_cancelled' else 'blocked',
'message': f'{error.code}: {error.message}' if resolving_bom else error.message}
paused = not safe
if not safe and error.code != 'operation_cancelled' and not stop_event.is_set():
try:
isolated = isolate_project(runtime, child, task_id, error)
except Exception as cleanup_error:
entry['message'] += '; 资源收尾未确认:' + str(cleanup_error)
isolated = False
if isolated:
isolated_tasks[task_id] = entry['message']
isolated_sources.update((source,task_id) for source in sources[task_id])
paused = False
except Exception as error:
entry = {'state': 'blocked', 'message': str(error)}
paused = True
if not stop_event.is_set():
try:
isolated = isolate_project(runtime, child, task_id, error)
except Exception as cleanup_error:
entry['message'] += '; 资源收尾未确认:' + str(cleanup_error)
isolated = False
if isolated:
isolated_tasks[task_id] = entry['message']
isolated_sources.update((source,task_id) for source in sources[task_id])
paused = False
state['entries'][task_id] = entry
record_task_outcome(child, task_id, entry['state'], entry['message'])
_save(path, state)
if (entry['state'] in {'completed', 'failed'} and task_id in checks.tasks
and not child.has_interrupted_quick_run(task_id)):
@@ -369,7 +426,8 @@ def execute_task_queue(service, task_ids: tuple[str, ...], *, stop_event: Event,
checks.after_task(task_id, execution_state='failed' if result.progress.failed_items else 'completed',
refresh=True, execution_summary={key: getattr(result.progress, key, None)
for key in ('saved_items', 'failed_items', 'no_match_items')})
retried = run_problem_pass(service, ids, token, stop_event=stop_event, runtime=runtime,
admitted_problem_tasks = tuple(item for item in ids if item not in isolated_tasks)
retried = run_problem_pass(service, admitted_problem_tasks, token, stop_event=stop_event, runtime=runtime,
on_result=capture_retry)
for task_id, result in retried.results.items():
state['entries'][task_id] = {'state': 'paused' if result.paused else 'failed' if result.progress.failed_items else 'completed',
@@ -416,3 +474,6 @@ def execute_task_queue(service, task_ids: tuple[str, ...], *, stop_event: Event,
checks.__exit__(None, None, None)
finally:
service._task_queue_feedback = None
runtime.stop_event = parent_control
if getattr(runtime, 'cad', None) is not None:
runtime.cad.stop_control = parent_control
@@ -9,12 +9,13 @@ STAGES = {
'snapshot_attributes': '记录原始属性', 'capture': '采集替换前快照',
'capture_untouched': '记录未选中器件状态', 'verify_untouched': '核验未选中器件',
'native_snapshot_read': '采集并核验器件完整状态',
'capture_semantic_state': '采集完整图页状态',
'before_native': '准备原生替换', 'native_replace': '执行原生替换',
'restore_preflight': '检查属性恢复条件',
'before_restore_transaction': '准备恢复属性',
'restore_attribute_fields': '恢复属性字段', 'restore_attributes': '恢复属性',
'verify_restoration': '核验恢复结果', 'verify_attribute_fields': '核验属性字段',
'save_and_close': '保存并关闭工程', 'verify_saved_project': '回读已保存工程',
'save_and_close': '保存并关闭工程', 'verify_saved_project': '回读工程并核对',
'awaiting_reconciliation': '当前结果待核验,已保留工程现场',
}
@@ -22,7 +23,20 @@ STAGES = {
def format_feedback(value, task_name, elapsed, stopping):
stage = STAGES.get(value.stage, '')
details = f'{stage}({value.stage_seconds:.1f} 秒)' if stage else value.message
prefix = '等待当前步骤保存核验后暂停 · ' if stopping else ''
reason = getattr(value, 'stopping_reason', '')
wait = getattr(value, 'wait_reason', '')
prefix = ({'user_pause': '已请求暂停,等待当前步骤完成',
'failure_cleanup': '执行失败,正在回收资源',
'inactivity': '当前步骤长时间无进展'}
.get(reason, '已请求暂停,等待当前步骤完成' if stopping else ''))
waits = {'native_call': '等待原生调用返回', 'save': '等待工程保存完成',
'readback': '等待工程重读核对', 'mentor_exit': '等待 Mentor 写进程退出',
'desktop_release': '等待后台桌面释放', 'reconciliation': '等待结果核对'}
if wait and (stopping or value.waiting_for_safe_completion):
prefix = ','.join(filter(None, (prefix, waits.get(wait, ''))))
if value.waiting_for_safe_completion:
prefix = '当前保存或核验尚未确认,未自动结束应用;可继续等待、导出诊断,或确认强制结束后核验 · '
prefix += ';最终结果尚未确认,可继续等待、导出诊断,或确认强制结束后核对'
if getattr(value, 'primary_error', ''):
details += ';原始错误:' + value.primary_error
prefix = prefix + ' · ' if prefix else ''
return f'{prefix}任务 {value.position}/{value.total}:{task_name} · {details} · 总计 {elapsed} 秒'
@@ -3,12 +3,20 @@ from dataclasses import dataclass
from ..models import TaskStatus
def task_display_status(task):
"""Legacy retryable failures retain running admission, but are not active work."""
if (task.status is TaskStatus.RUNNING
and task.summary == 'current-sheet quick-run queue completed with retryable failures'):
return TaskStatus.FAILED
return task.status
def pending_batch_tasks(tasks, group_id):
"""Use the persisted batch order for both execution and its UI controls."""
if not group_id:
return ()
return tuple(sorted((task for task in tasks if task.group_id == group_id
and task.status not in {TaskStatus.DELETED, TaskStatus.COMPLETED}
and task.status not in {TaskStatus.DELETED, TaskStatus.COMPLETED, TaskStatus.BLOCKED}
and getattr(task, 'summary', '') != 'current-sheet quick-run queue completed with retryable failures'),
key=lambda task: task.group_position))
@@ -34,7 +42,8 @@ def execution_state(context, *, busy=False, dirty=False):
if not project or task.status is TaskStatus.DELETED:
return ExecutionState(message='任务已删除,可从已删除列表恢复。')
editable = task.status in {TaskStatus.DRAFT, TaskStatus.READY, TaskStatus.FAILED} and not context.has_evidence and not busy
retry = context.phase == 'retry' and not context.interrupted and not busy and not dirty
retry = (context.phase == 'retry' and task.status is not TaskStatus.BLOCKED
and not context.interrupted and not busy and not dirty)
requests = getattr(context, 'pending_bom', ())
deferred = bool(requests)
from ..bom_pending import retained_bom_rows
@@ -44,7 +53,7 @@ def execution_state(context, *, busy=False, dirty=False):
execute = (pending and task.status in {TaskStatus.DRAFT, TaskStatus.READY, TaskStatus.RUNNING, TaskStatus.PAUSED}
and project.available and not context.interrupted and not busy and not dirty)
text = '核验并执行' if deferred else '继续当前任务' if task.status is TaskStatus.PAUSED else '执行当前任务'
message = ('结果待核验,请先恢复中断步骤。' if context.interrupted else
message = ('结果待核验,请先恢复中断步骤。' if context.interrupted or task.status is TaskStatus.BLOCKED else
'位号计划有未保存编辑,请先保存。' if dirty else
'正在处理当前步骤,暂停会等待安全收尾。' if busy else
'任务已完成。可查看原计划和日志;点击“开始新一轮”归档本批已完成任务,再采集当前料号。' if task.status is TaskStatus.COMPLETED else
@@ -54,5 +63,6 @@ def execution_state(context, *, busy=False, dirty=False):
'BOM 表格已保存;确认后仅核验当前工程位号,再执行替换。' if deferred else
'先填写旧、新料号,再采集并勾选位号。' if not pending else
'配置就绪,可执行当前任务或按顺序执行本批任务。')
return ExecutionState(editable, execute, retry, context.interrupted and not busy and not dirty, text, message,
return ExecutionState(editable, execute, retry,
(context.interrupted or task.status is TaskStatus.BLOCKED) and not busy and not dirty, text, message,
task.status is TaskStatus.COMPLETED and not context.interrupted and not busy and not dirty)
@@ -423,9 +423,10 @@ class TrialTaskCenterWindow(MaterialMainWindow):
def _refresh_task_list(self):
deleted = self.task_list.recycle_button.isChecked()
def received(tasks):
self._known_task_groups.update((task.task_id, task.group_id) for task in tasks)
self._tasks = tasks
self.task_list.populate(tasks, selected_task_id=self._task_id)
if tasks != self._tasks or not self.task_list.count.text():
self._known_task_groups.update((task.task_id, task.group_id) for task in tasks)
self._tasks = tasks
self.task_list.populate(tasks, selected_task_id=self._task_id)
self._update_gates()
self._task_query.request(lambda control: self.service.tasks(status=TaskStatus.DELETED, read_control=control)
if deleted else self.service.tasks(read_control=control), received, self._show_error, cancellable=True)
@@ -529,7 +530,7 @@ class TrialTaskCenterWindow(MaterialMainWindow):
self.run_panel.retry_button.setVisible(bool(context and context.phase == 'retry'))
self.run_panel.retry_button.setEnabled(state.retry and not resources_pending)
self.run_panel.problems_button.setEnabled(bool(context and not busy))
self.run_panel.recover_button.setVisible(bool(context and context.interrupted))
self.run_panel.recover_button.setVisible(bool(context and (context.interrupted or context.task.status is TaskStatus.BLOCKED)))
self.run_panel.recover_button.setEnabled(state.recover and not resources_pending)
self.run_panel.new_round_button.setVisible(bool(context and context.task.status is TaskStatus.COMPLETED))
self.run_panel.new_round_button.setEnabled(state.new_round and not group_unsaved)
@@ -597,7 +598,8 @@ class TrialTaskCenterWindow(MaterialMainWindow):
self.run_panel.execution_status.setText('正在检查后台资源' if not self._resource_status.known else
'资源状态读取失败' if self._resource_status.error is not None else
'资源待清理' if resources_pending else '任务读取失败' if read_failed else
'结果待核验' if context and context.interrupted else '空闲')
'结果待核验' if context and (context.interrupted or context.task.status is TaskStatus.BLOCKED) else
'旧桌面已隔离,可继续执行' if self._resource_status.records else '空闲')
def _group_has_unsaved_plans(self):
if not self._context:
@@ -1037,16 +1039,22 @@ class TrialTaskCenterWindow(MaterialMainWindow):
cleanup_finished = True
start_force_if_ready()
def forced(remaining):
from ..batch_resources import resource_blockers
self._resource_status.refresh()
self._set_operator_message('仍有资源未能确认释放,请保留诊断。' if remaining else
self._set_operator_message('仍有资源未能确认释放,请保留诊断。' if resource_blockers(remaining) else
'自有写入进程已退出,残留桌面已隔离(未确认释放);相关工程仍须核验。' if remaining else
'自有残留资源已释放;相关工程仍须核验后恢复。', 'warning')
self._update_gates()
def done(pending):
nonlocal force_confirmed
from ..batch_resources import resource_blockers
warnings = tuple(row for row in pending if row.get('blocking') is False)
pending = resource_blockers(pending)
hints = tuple(dict.fromkeys(row['resolution_hint'] for row in pending if row.get('resolution_hint')))
self._set_operator_message(
'\n'.join(hints) if hints else '后台资源仍待清理,请保留诊断记录后重试。' if pending else '后台资源已释放,可以继续。',
'warning' if pending else 'success')
'\n'.join(hints) if hints else '后台资源仍待清理,请保留诊断记录后重试。' if pending else
'旧桌面仍存在,已隔离;自有写进程已退出,可以继续执行。' if warnings else '后台资源已释放,可以继续。',
'warning' if pending or warnings else 'success')
if pending:
self._dsh_blockers.report('resource-release', '', 'batch_resources_pending',
'后台资源清理完成后仍有资源未确认释放。', {'resources': list(pending)[:20]}, cleanup_allowed=True)
@@ -1187,7 +1195,7 @@ class TrialTaskCenterWindow(MaterialMainWindow):
if not self.run_panel.recover_button.isEnabled():
return
if QMessageBox.question(self, '核验并恢复',
'先备份工程和证据,再打开工程核验;如需补齐属性,会保存并重开回读,不会重复调用原生替换。请确保待处理工程未在其他 CAD 会话中打开。',
'共享数据源依赖导致隔离、且从未派发的任务,将核验依赖与原计划后解除隔离,不打开 CAD。其他中断任务先备份工程和证据,再打开核验;如需补齐属性,会保存并重开回读,不会重复调用原生替换。请确保待处理工程未在其他 CAD 会话中打开。',
QMessageBox.StandardButton.Yes | QMessageBox.StandardButton.No,
QMessageBox.StandardButton.No) != QMessageBox.StandardButton.Yes:
return
@@ -1223,6 +1231,8 @@ class TrialTaskCenterWindow(MaterialMainWindow):
def _update_quick_run_elapsed(self):
self._update_gates()
if self._batch_stop is not None and not self._task_query.busy:
self._refresh_task_list()
if self._batch_stop is not None and not self._feedback_query.busy:
task_id, started = self._active_quick_run_task_id, self._run_started_at
run_id = self._dsh_run_id
@@ -1459,6 +1469,7 @@ class TrialTaskCenterWindow(MaterialMainWindow):
# If it fails, release the execution controls just as
# the service failure path does.
self._finish_run()
self._refresh_task_list()
self.operation_state = self.operation_state.finish(error)
self._show_error(error, owner_task_id=owner, operation_area=operation_area)
def failed(error):
@@ -1470,6 +1481,8 @@ class TrialTaskCenterWindow(MaterialMainWindow):
if self._close_saving or not (isinstance(error, TaskCenterError) and error.code == 'operation_cancelled'):
self._close_pending = False
self._finish_run()
if execution:
self._refresh_task_list()
if isinstance(error, TaskCenterError) and error.code == 'operation_cancelled':
self._set_operator_message(self.operation_state.message)
return
@@ -29,7 +29,8 @@ class ResourceStatus(QObject):
@property
def blocked(self):
return not self.known or self.error is not None or bool(self.records)
from ..batch_resources import resource_blockers
return not self.known or self.error is not None or bool(resource_blockers(self.records))
def refresh(self):
if self._closed:
@@ -12,6 +12,7 @@ from PySide6.QtWidgets import (QAbstractItemView,QComboBox,QHBoxLayout,QLabel,QL
QListWidgetItem,QPushButton,QVBoxLayout,QWidget,QToolButton,QMenu,
QStyledItemDelegate,QStyleOptionViewItem,QStyle,QApplication)
from .command_menu import CommandMenu
from .execution_state import task_display_status
class TaskItemDelegate(QStyledItemDelegate):
@@ -80,10 +81,13 @@ class TaskList(QWidget):
self.count = QLabel(objectName='taskCount')
self.filter = QComboBox(objectName="taskFilter")
for label, key, picture in (('全部任务', 'all', 'grid'), ('未完成', 'unfinished', 'clock'),
('已完成', 'completed', 'check-circle'), ('失败', 'failed', 'warning')):
('待开始', 'pending', 'clock'), ('执行中', 'running', 'clock'),
('已暂停', 'paused', 'clock'), ('已完成', 'completed', 'check-circle'),
('待核验', 'blocked', 'warning'),
('失败 / 待重试', 'failed', 'warning')):
self.filter.addItem(icon(picture), label, key)
self.filter.setAccessibleName('任务状态分类')
self.filter.setToolTip('未完成包括待配置、待执行、执行中、已暂停和失败的任务;分类只筛选显示。')
self.filter.setToolTip('待开始包括待配置和待执行;失败包括已收尾的待重试任务。未完成包括所有尚未完成的任务;分类只筛选显示。')
self.list = QListWidget(objectName="taskList")
self.list.setItemDelegate(TaskItemDelegate(self.list))
self.list.setSelectionMode(QAbstractItemView.SelectionMode.ExtendedSelection)
@@ -163,9 +167,17 @@ class TaskList(QWidget):
return str(item.data(Qt.ItemDataRole.UserRole)) if item is not None else None
def populate(self, tasks: Iterable[object], *, selected_task_id: str | None = None) -> None:
preserve_selection = selected_task_id in (None, self.current_task_id())
selected_ids = {item.data(Qt.ItemDataRole.UserRole) for item in self.list.selectedItems()}
self._selected_task_id = selected_task_id or self.current_task_id() or self._selected_task_id
self._tasks = tuple(tasks)
self._render()
if preserve_selection:
with QSignalBlocker(self.list):
for row in range(self.list.count()):
item = self.list.item(row)
item.setSelected(item.data(Qt.ItemDataRole.UserRole) in selected_ids)
self._update_selection_state()
def select_task(self, task_id: str) -> None:
if task_id != self._selected_task_id:
@@ -209,8 +221,9 @@ class TaskList(QWidget):
category = self.filter.currentData()
if not self.recycle_button.isChecked() and category != 'all':
tasks = [task for task in tasks if
(task.status.value not in {'completed', 'deleted'} if category == 'unfinished'
else task.status.value == category)]
(task_display_status(task).value not in {'completed', 'deleted'} if category == 'unfinished'
else task_display_status(task).value in {'draft', 'ready'} if category == 'pending'
else task_display_status(task).value == category)]
mode = self.sort.currentData()
if mode.startswith('name_'):
tasks.sort(key=lambda task: tuple(int(part) if part.isdigit() else part
@@ -230,9 +243,9 @@ class TaskList(QWidget):
for task in tasks:
group_id = getattr(task, "group_id", None)
prefix = f"{getattr(task, 'group_position', 0) + 1:02d}. " if group_id else ""
status = getattr(getattr(task, 'status', None), 'value', '')
status = task_display_status(task).value
labels = {'draft': '待配置', 'ready': '待执行', 'running': '执行中', 'paused': '已暂停',
'completed': '已完成', 'failed': '失败', 'deleted': '已删除'}
'completed': '已完成', 'failed': '失败', 'blocked': '待核验', 'deleted': '已删除'}
item = QListWidgetItem(prefix + str(getattr(task, "name")) + f"\n{labels.get(status, status)}")
item.setData(Qt.ItemDataRole.UserRole, str(getattr(task, "task_id")))
item.setIcon(self._project_icon)
@@ -37,14 +37,28 @@ def _fields(line):
return tuple(_decode_vbs_field(field) for field in line.split('\t'))
def _validated(root, row, outcome):
def _validated(root, row, outcome, *, project_commit=None):
raw, _, snapshot_path = _paths(root, row)
state, code, message = outcome
if state not in {'saved_ok', 'restored_ok'}:
raise ValueError('Value report requires a verified replacement terminal state')
status_bytes = _read(internal_path(root) / 'status.tsv')
terminal = _fields(status_bytes.decode('utf-8-sig').splitlines()[-1])
if terminal != (row['job_id'], state, code, message):
if project_commit is not None:
from .execution_proof import proof_intact
from .path_access import canonical_path
if (code != 'project_committed' or not proof_intact(project_commit)
or canonical_path(project_commit['project_path']) != canonical_path(row['project_path'])
or project_commit['outcomes'].get(row['job_id']) != state):
raise ValueError('Value report has no matching intact project commit')
marker = internal_path(project_commit['files'][0]['path'])
lines = [_fields(line) for line in _read(marker).decode('utf-8-sig').splitlines() if line]
if (lines[0] != (project_commit['session_token'],row['project_path'],'project_verified')
or [fields[0] for fields in lines[1:]] != project_commit['jobs']
or dict(lines[1:]) != project_commit['outcomes']):
raise ValueError('Value report project commit marker differs')
terminal = (row['job_id'],state,code,message)
elif terminal != (row['job_id'], state, code, message):
raise ValueError('Value report does not belong to the current terminal job')
raw_bytes, snapshot_bytes = _read(raw), _read(snapshot_path)
snapshot = snapshot_bytes.decode('utf-8-sig')
@@ -74,8 +88,10 @@ def _validated(root, row, outcome):
if row.get('target_scope') == 'targets':
selected = {parts[0]: parts[1] for selector in selectors.split('\x1e')
if len(parts := selector.split('\x1f')) == 3}
if components != selected:
raise ValueError('Value snapshot UID set differs from frozen command')
expected_refs = {ref.strip().casefold() for ref in selected.values()}
if (not selected or len(expected_refs) != len(selected)
or {ref.strip().casefold() for ref in components.values()} != expected_refs):
raise ValueError('Value snapshot physical reference set differs from frozen command')
values, observations, seen, ordinals = [], [], set(), set()
names = {}
for line in lines[1:]:
@@ -117,14 +133,16 @@ def _validated(root, row, outcome):
value_readback_job_id=row['job_id'], project_path=row['project_path'],
schematic_name=row.get('schematic_name', ''), sheet_number=row.get('sheet_number', ''),
attribute_observations=observations, attribute_ambiguities=ambiguities)
binding = dict(command=dict(row), terminal=list(terminal), status_sha256=_digest(status_bytes),
binding = dict(command=dict(row), terminal=list(terminal), status_sha256=None if project_commit else _digest(status_bytes),
snapshot_sha256=_digest(snapshot_bytes), report_sha256=_digest(raw_bytes))
if project_commit is not None:
binding['project_commit'] = project_commit
return details, binding
def capture_value_readback(root, row, outcome):
def capture_value_readback(root, row, outcome, *, project_commit=None):
try:
details, binding = _validated(root, row, outcome)
details, binding = _validated(root, row, outcome, project_commit=project_commit)
_, saved, _ = _paths(root, row)
write_record(saved, dict(schema=2, binding=binding, details=details))
return details
@@ -136,9 +154,10 @@ def capture_value_readback(root, row, outcome):
def load_value_readback(root, row, outcome):
try:
details, binding = _validated(root, row, outcome)
_, saved, _ = _paths(root, row)
persisted = json.loads(_read(saved))
details, binding = _validated(root, row, outcome,
project_commit=persisted.get('binding',{}).get('project_commit'))
if persisted != dict(schema=2, binding=binding, details=details):
raise ValueError('persisted Value evidence binding changed')
return details
@@ -362,6 +362,7 @@ def run_monitored_process(command: tuple[str, ...], idle_timeout: float, *, abor
stop_readers = threading.Event()
output = [bytearray(), bytearray()]
startup_pause = None
prewrite_pause = None
def read_stream(stream, channel):
try:
@@ -379,7 +380,7 @@ def run_monitored_process(command: tuple[str, ...], idle_timeout: float, *, abor
stream.close()
def receive(channel, line):
nonlocal startup_pause
nonlocal startup_pause, prewrite_pause
if channel == 0 and watch.accept(line, time.monotonic()):
_publish_progress(progress_path, watch.snapshot())
elif channel == 0 and line.startswith(b'DX_STARTUP_PAUSED\t'):
@@ -399,6 +400,29 @@ def run_monitored_process(command: tuple[str, ...], idle_timeout: float, *, abor
'project_path': request[1], 'sequence': watch.sequence}
except (OSError, UnicodeError, ValueError):
pass
elif channel == 0 and line.startswith(b'DX_PREWRITE_PAUSED\t'):
try:
from .quick_run import _decode_vbs_field
from .path_access import canonical_path
fields = line.decode('ascii').strip().split('\t')
request_data = (root / 'project-session.request').read_bytes()
request = request_data.decode('utf-8-sig').strip().split('\t')
original = session_request.decode('utf-8-sig').strip().split('\t') if session_request else []
if (len(fields) == 6 and fields[:3] == ['DX_PREWRITE_PAUSED', token, job_id]
and len(request) == 3 and len(original) == 3 and request[:2] == original[:2]
and fields[3:5] == [request[0], request[2]] and re.fullmatch('[0-9a-f]{32}', request[0])
and re.fullmatch(r'[A-Za-z0-9_.-]{1,128}', request[2])
and int(fields[5]) == watch.sequence and watch.sequence > 0
and abort_event is not None and abort_event.is_set() and watch.stop_requested):
marker = (root / ('project-pause-' + request[0] + '.tsv')).read_bytes()
values = marker.decode('utf-8-sig').strip().split('\t')
if (len(values) == 4 and values[0] == request[0] and values[2:] == [request[2], 'prewrite_closed']
and canonical_path(_decode_vbs_field(values[1])) == canonical_path(request[1])):
prewrite_pause = {'monitor_token': token, 'session_token': request[0], 'job_id': request[2],
'project_path': request[1], 'sequence': watch.sequence,
'request_bytes': request_data, 'marker_bytes': marker}
except (OSError, UnicodeError, ValueError):
pass
elif len(output[channel]) < 1024 * 1024:
output[channel].extend(line[:1024 * 1024 - len(output[channel])])
@@ -545,6 +569,15 @@ def run_monitored_process(command: tuple[str, ...], idle_timeout: float, *, abor
or not _worker_tree_exited(process, owned)):
raise WorkerMonitorError('startup pause lacks correlated no-project acknowledgement or worker exit')
result.startup_pause = dict(startup_pause, worker_tree_exited=True)
if process.returncode == 75:
if (prewrite_pause is None or prewrite_pause['sequence'] != watch.sequence
or result.stderr.strip() or not _worker_tree_exited(process, owned)
or (root / 'project-session.request').read_bytes() != prewrite_pause['request_bytes']
or (root / ('project-pause-' + prewrite_pause['session_token'] + '.tsv')).read_bytes()
!= prewrite_pause['marker_bytes']):
raise WorkerMonitorError('prewrite pause lacks correlated closed-project acknowledgement or worker exit')
result.prewrite_pause = {key: value for key, value in prewrite_pause.items()
if key not in {'request_bytes', 'marker_bytes'}} | {'worker_tree_exited': True}
return result
except Exception as error:
if process is not None:
@@ -9,21 +9,14 @@ from uuid import uuid4
from .batch_resources import write_record
from .errors import TaskCenterError
from .path_access import canonical_path, internal_path
from .stop_control import StopControl, protect_interruption
from .stop_control import ProjectStopControl, protect_interruption
from .project_isolation import write_sources, record_task_outcome, block_dependency, dependency_readmitted
_LOCK = Lock()
_TERMINAL = {'completed', 'failed', 'skipped', 'blocked'}
class _ProjectControl(StopControl):
def __init__(self, parent):
super().__init__()
self.parent = parent
if hasattr(parent, 'force_event'):
self.force_event = parent.force_event
def is_set(self):
return super().is_set() or self.parent.is_set()
_ProjectControl = ProjectStopControl
@contextmanager
@@ -68,7 +61,8 @@ def _released_after_recheck(root, task_id, row):
journal = json.loads(internal_path(root / 'batch-resources' / batch_id / 'supervisor.json').read_text(encoding='utf-8'))
return (valid_journal(journal, batch_id) and journal['task_ids'] == [task_id]
and journal['state'] == 'finished' and journal.get('process')
and _process_gone(journal['process']) and not pending_resources(root) and not pending_supervisors(root))
and _process_gone(journal['process']) and not any(row.get('blocking', True) for row in pending_resources(root))
and not any(row.get('blocking', True) for row in pending_supervisors(root)))
except (OSError, ValueError, KeyError, TypeError):
return False
@@ -88,6 +82,7 @@ def execute_workspace_projects(service, task_ids, *, stop_event, queue_id=None,
for rows in projects}) != len(ids):
raise TaskCenterError('project_binding', '每个任务必须对应不同的单个工程。')
hashes = {item: _plan_hash(children[item], item) for item in ids}
sources = {item: write_sources(rows[0].path) for item, rows in zip(ids, projects)}
if expected_plan_hashes is not None and hashes != expected_plan_hashes:
raise TaskCenterError('assistant_stale', '执行计划已变化,请重新预检。')
token = queue_id or checkpoint_id or uuid4().hex
@@ -115,6 +110,9 @@ def execute_workspace_projects(service, task_ids, *, stop_event, queue_id=None,
for item in ids}}
write_record(path, state)
isolated_sources = state.setdefault('isolated_sources', {})
dependency_tasks = state.setdefault('dependency_tasks', {})
def result(paused=False, current=None):
verification = {item: state['entries'][item]['netlist_verification'] for item in ids
if state['entries'][item].get('netlist_verification')}
@@ -140,17 +138,25 @@ def execute_workspace_projects(service, task_ids, *, stop_event, queue_id=None,
or outcome.entries[0].state not in {'pending', 'running', 'paused', *_TERMINAL}):
raise TaskCenterError('task_queue_state', '工程监督进程返回了不同的队列身份。')
row.update(state=outcome.entries[0].state, message=outcome.entries[0].message, released=True)
if row['state'] == 'blocked':
record_task_outcome(children[item], item, 'blocked', row['message'])
isolated_sources.update((source, item) for source in sources[item])
verification = (outcome.netlist_verification or {}).get(item)
if verification:
row['netlist_verification'] = verification
write_record(path, state)
return outcome.paused and row['state'] != 'blocked'
except TaskCenterError as error:
released = getattr(error, 'resources_released', False) is True
from .batch_resources import resource_blockers
released = ((getattr(error, 'resources_released', False) is True
or getattr(error, 'owned_writers_exited', False) is True)
and not resource_blockers(service.pending_batch_resources() if hasattr(service, 'pending_batch_resources') else ()))
row.update(state='blocked', message=error.message, released=released)
row['supervisor_batch_id'] = getattr(service, '_last_supervisor_batch_id', None)
evidence = getattr(getattr(children[item], '_quick_run_preparer', None), 'data_root', root / 'tasks' / item)
protect_interruption(Path(evidence) / 'quick-run', 'project_supervisor_failed: ' + error.message)
record_task_outcome(children[item], item, 'blocked', error.message)
isolated_sources.update((source, item) for source in sources[item])
if released:
from .batch_supervisor import reconcile_supervisors
try:
@@ -165,9 +171,31 @@ def execute_workspace_projects(service, task_ids, *, stop_event, queue_id=None,
raise
try:
if state['phase'] == 'finished' and any(dependency_readmitted(children[item], item, hashes[item])
for item in tuple(dependency_tasks)):
state['phase'] = 'primary'
if state['phase'] == 'primary':
for item in ids:
row = state['entries'][item]
readmitted = False
if item in dependency_tasks:
try:
readmitted = dependency_readmitted(children[item], item, hashes[item])
except TaskCenterError:
pass
if readmitted:
row.update(state='pending', message='共享依赖已核验,恢复未派发原计划。')
dependency_tasks.pop(item)
conflict = next((isolated_sources[source] for source in sources[item]
if source in isolated_sources and isolated_sources[source] != item), None)
if conflict is not None and not readmitted and row['state'] not in {'completed', 'failed', 'skipped'}:
message = f'与待核验工程 {conflict} 共享写入源,已隔离;未派发替换。'
block_dependency(root, children[item], item, conflict,
sources[item] & sources[conflict], hashes[item], token)
dependency_tasks[item] = conflict
row.update(state='blocked', message=message, released=True, problems_done=True)
write_record(path, state)
continue
if row['state'] in _TERMINAL:
if row['state'] == 'blocked' and not row['released']:
if not _released_after_recheck(root, item, row):
@@ -1,18 +1,14 @@
import importlib.util
import inspect
from pathlib import Path
import subprocess
import tempfile
from types import SimpleNamespace
import unittest
from unittest.mock import call, patch
ROOT = Path(__file__).resolve().parents[2]
RUNTIME = ROOT / "integration" / "dxdesigner795"
SPEC = importlib.util.spec_from_file_location(
"native", ROOT / "src" / "dx_productivity" / "trial_task_center" / "native_helper.py"
)
native = importlib.util.module_from_spec(SPEC)
SPEC.loader.exec_module(native)
from dx_productivity.trial_task_center import native_helper as native
FAKES = r"""
Class TestPoint
@@ -107,7 +103,7 @@ End Sub
class NativeTests(unittest.TestCase):
def test_native_command_does_not_use_physical_mouse_fallback(self):
source = inspect.getsource(native.run_native)
source = inspect.getsource(inspect.unwrap(native.run_native))
self.assertIn("replacement_command_from_resources(executable)", source)
self.assertNotIn("win32api.SetCursorPos", source)
@@ -115,20 +111,24 @@ class NativeTests(unittest.TestCase):
self.assertNotIn("win32gui.SetCursorPos(screen_point)", source)
def test_snapshot_batch_is_reselected_before_the_strict_selection_gate(self):
app = object()
api = object()
metadata = ["M"]
first = type("Component", (), {"UID": "wanted"})()
other = type("Component", (), {"UID": "other"})()
first = SimpleNamespace(UID='wanted', Refdes='R1', Selected=False)
other = SimpleNamespace(UID='other', Refdes='R2', Selected=True)
first._oleobj_, other._oleobj_ = first, other
app = SimpleNamespace(ActiveView=SimpleNamespace(Query=lambda kind, scope:
[component for component in (first, other) if component.Selected]))
api = SimpleNamespace(IVdComp=lambda value:value)
records = [["C", "wanted"]]
with patch.object(native, "check_context", side_effect=([first, other], [first, other])) as check:
with patch.object(native, "selection") as select:
with patch.object(native, "check_context", return_value=[first, other]) as check:
with patch.object(native, "selection", side_effect=lambda component, selected:setattr(component,'Selected',selected)) as select, \
patch.object(native, "check_active_context"):
result = native.select_snapshot_batch(app, api, metadata, records)
self.assertEqual(result, [first, other])
self.assertEqual(check.call_args_list[0], call(app, api, metadata, records, require_selection=False))
self.assertEqual(check.call_args_list[1], call(app, api, metadata, records))
self.assertEqual(select.call_args_list, [call(first, True), call(other, False)])
self.assertTrue(first.Selected)
self.assertFalse(other.Selected)
def test_snapshot_escape_format(self):
self.assertEqual(native.decode("a%0009b%000Ac%0025"), "a\tb\nc%")
@@ -73,10 +73,11 @@ Class SessionApp
End Function
End Class
Class SessionDocuments
Public Owner, OpenCount, SheetCount
Public Owner, OpenCount, SheetCount, DocumentCount
Private Sub Class_Initialize()
OpenCount = 0
SheetCount = 1
DocumentCount = 1
End Sub
Public Function GetAvailableSchematics()
Dim Names
@@ -94,13 +95,14 @@ Class SessionDocuments
End Function
Public Function Open(ByVal Schematic, ByVal Sheet)
OpenCount = OpenCount + 1
DocumentCount = 1
Set Owner.ActiveView = Owner
Set Open = Me
End Function
Public Sub Activate()
End Sub
Public Property Get Count
Count = 1
Count = DocumentCount
End Property
Public Function Item(ByVal Index)
If Index <> 1 Then Err.Raise 51000,"Test","bad document index"
@@ -108,6 +110,7 @@ Class SessionDocuments
End Function
Public Sub Close(ByVal SaveChanges, ByVal FileName)
If SaveChanges Then Owner.SaveCount = Owner.SaveCount + 1
DocumentCount = DocumentCount - 1
Set Owner.ActiveView = Nothing
End Sub
End Class
@@ -141,6 +144,9 @@ def run_session(tmp_path, body):
target.touch()
setup = f'''
Set QP_Fso = CreateObject("Scripting.FileSystemObject")
Set QP_SessionItems=CreateObject("Scripting.Dictionary")
Set QP_SessionSheets=CreateObject("Scripting.Dictionary")
Set QP_SessionDocuments=CreateObject("Scripting.Dictionary")
QP_Root = "{tmp_path}"
Dim Target, Attached
Target = "{target}"
+21 -6
View File
@@ -90,6 +90,7 @@ Class ValueApp
End Function
Public Sub Close(ByVal SaveChanges,ByVal FileName)
CloseCount=CloseCount+1
DocumentCount=DocumentCount-1
If SaveChanges Then
SaveCalled=True
SaveCount=SaveCount+1
@@ -149,6 +150,9 @@ End Function
SETUP = r'''
Dim Comp, Pin, Attr, Ignored, Command, Snapshot, SnapshotPath, Failure, Saved
Set QP_Fso=CreateObject("Scripting.FileSystemObject")
Set QP_SessionItems=CreateObject("Scripting.Dictionary")
Set QP_SessionSheets=CreateObject("Scripting.Dictionary")
Set QP_SessionDocuments=CreateObject("Scripting.Dictionary")
QP_Root="__ROOT__"
QP_CurrentJobId="value-job"
Set QP_App=New ValueApp
@@ -220,7 +224,7 @@ Require QP_App.UndoCount=0,"recovery undid prior CAD work"
''')
@pytest.mark.parametrize('change', ['Comp.UID="imposter"', 'Comp.Refdes="R99"'])
@pytest.mark.parametrize('change', ['Comp.Refdes="R99"'])
def test_restore_rejects_changed_component_identity_before_any_write(tmp_path, change):
run_vbs(tmp_path, 'part_number', '''
On Error Resume Next
@@ -233,6 +237,16 @@ Require QP_App.TransactionCount=0,"identity failure must precede CAD writes"
''', after=change)
def test_restore_accepts_uid_churn_without_changing_reference_or_replaying_replace(tmp_path):
run_vbs(tmp_path, 'part_number', '''
QPRestoreAfterOperatorReplace Command,SnapshotPath
Require QP_App.TransactionCount=1,"restoration did not use exactly one transaction"
Require Comp.Refdes="R1","reference identity changed"
Require Comp.FindAttribute("Value").InstanceValue="NEW","Value policy was not verified"
Require QP_App.UndoCount=0,"UID churn replayed or undid the native replacement"
''', after='Comp.UID="changed-native-uid"')
def _batch_readback_fixture(routines, *, fault=''):
# Exercise real restore/finish/guards, replacing only the external open and
# command transport with an in-memory CAD-shaped application.
@@ -242,6 +256,7 @@ End Function''', routines)
routines = re.sub(r'(?ms)^Sub QPEnsureProject\([^\n]*\n.*?^End Sub', '''Sub QPEnsureProject(ByRef App, ByVal TargetPath)
App.ProjectPath=TargetPath
App.CloseCount=0
App.DocumentCount=1
QP_OwnedProject=True
QP_OwnedProjectPath=TargetPath
''' + ('Set Attr=Other.FindAttribute("Value"): Attr.Value="DRIFT"' if fault == 'after_reopen' else '') + '''
@@ -258,7 +273,7 @@ def test_save_then_project_close_refusal_retains_save_boundary(tmp_path):
Dim FullChecks
FullChecks=0
Command.Add "project_path",QPProjectPath()
QP_BatchMode=True: QP_SessionMode=True
QP_BatchMode=True: QP_SessionMode=False
QP_SessionToken=String(32,"b")
QP_OwnedProject=True: QP_OwnedProjectPath=QPProjectPath()
QPRestoreAfterOperatorReplace Command,SnapshotPath
@@ -274,7 +289,7 @@ QP_App.RefuseClose=False
QPDiscardFailedBatchProject CloseError
Require Len(CloseError)>0,"uncertain save must not discard the open project"
Require Len(QPProjectPath())>0,"failed saved project was silently discarded"
Require Not QP_Fso.FileExists(QP_Fso.BuildPath(QP_Root,"project-session-result.tsv")),"failure published success"
Require Not QP_Fso.FileExists(QP_Fso.BuildPath(QP_Root,"status.tsv")),"failure published success"
''', routines_transform=_batch_readback_fixture)
@@ -284,7 +299,7 @@ def test_owned_batch_checks_full_state_at_save_and_disk_boundaries(tmp_path, fau
Dim FullChecks
FullChecks=0
Command.Add "project_path",QPProjectPath()
QP_BatchMode=True: QP_SessionMode=True
QP_BatchMode=True: QP_SessionMode=False
QP_SessionToken=String(32,"b")
QP_OwnedProject=True: QP_OwnedProjectPath=QPProjectPath()
QPRestoreAfterOperatorReplace Command,SnapshotPath
@@ -298,10 +313,10 @@ On Error GoTo 0
Require Failure=0,"valid batch failed"
Require FullChecks=2,"expected one full check before save and one after reopen"
Require QP_App.SaveCalled,"valid batch did not save"
Require InStr(QPReadUtf8(QP_Fso.BuildPath(QP_Root,"project-session-result.tsv")),"saved_ok")>0,"no verified result"
Require InStr(QPReadUtf8(QP_Fso.BuildPath(QP_Root,"status.tsv")),"saved_ok")>0,"no verified result"
''' if not fault else '''
Require Failure<>0,"unselected drift passed the commit guard"
Require Not QP_Fso.FileExists(QP_Fso.BuildPath(QP_Root,"project-session-result.tsv")),"failure published a completion"
Require Not QP_Fso.FileExists(QP_Fso.BuildPath(QP_Root,"status.tsv")),"failure published a completion"
Require QP_App.SaveCalled=''' + ('True' if fault == 'after_reopen' else 'False') + ''',"wrong save boundary"
Require QP_BatchSaveCommitted=''' + ('True' if fault == 'after_reopen' else 'False') + ''',"saved uncertainty marker is wrong"
'''), after='''
+17 -13
View File
@@ -104,11 +104,11 @@ def test_no_match_is_recorded_as_information_not_warning(tmp_path):
assert executed[0]['details']['state'] == 'no_match'
def test_discovery_inventory_completes_definite_misses_without_reopening_projects(tmp_path):
def test_discovery_inventory_records_missing_targets_without_reopening_projects(tmp_path):
service, task, calls = setup_queue(tmp_path, ('no_match', 'no_match'))
preparer = service._quick_run_preparer
preparer._sheet_discoverer = None
preparer._discover_project_sheets = lambda _project, _parts: _ProjectDiscovery(
preparer._discover_project_sheets = lambda _project, _parts, **_options: _ProjectDiscovery(
(('Main', '1'),),
{('Main', '1'): frozenset()},
)
@@ -116,16 +116,17 @@ def test_discovery_inventory_completes_definite_misses_without_reopening_project
result = run_queue(service, task)
assert result.progress.queue_complete
assert result.progress.no_match_items == 2
assert result.progress.failed_items == 2
assert result.progress.no_match_items == 0
assert calls == []
assert all(project.summary == 'quick_run_result:0:1:saved' for project in service.projects(task.task_id))
assert all(project.summary.startswith('quick_run_result:0:0:failed') for project in service.projects(task.task_id))
events = [
json.loads(line)
for line in (tmp_path / 'data/tasks' / task.task_id / 'events.jsonl').read_text(encoding='utf-8').splitlines()
]
prefiltered = [event for event in events if event['operation'] == 'quick_run_inventory_prefiltered']
assert len(prefiltered) == 1
assert prefiltered[0]['details']['prefiltered_items'] == 2
assert not any(event['operation'] == 'quick_run_executed' for event in events)
status = (tmp_path / 'data/quick-run/status.tsv').read_text(encoding='utf-8')
assert status.count('\tbatch_skipped\tinventory_empty\t') == 2
def test_discovery_inventory_executes_present_sources_and_skips_only_absent_sources(tmp_path):
@@ -133,9 +134,11 @@ def test_discovery_inventory_executes_present_sources_and_skips_only_absent_sour
service.add_rule(task.task_id, 'ABSENT', 'OTHER')
preparer = service._quick_run_preparer
preparer._sheet_discoverer = None
preparer._discover_project_sheets = lambda _project, _parts: _ProjectDiscovery(
from dx_productivity.trial_task_center.target_plan import ComponentTarget
preparer._discover_project_sheets = lambda _project, _parts, **_options: _ProjectDiscovery(
(('Main', '1'),),
{('Main', '1'): frozenset({'old'})},
(ComponentTarget(_project, 'Main', '1', 'u1', 'R1', 'OLD', '10K'),),
)
def worker(command, timeout):
del timeout
@@ -152,7 +155,8 @@ def test_discovery_inventory_executes_present_sources_and_skips_only_absent_sour
assert result.progress.queue_complete
assert result.progress.saved_items == 1
assert result.progress.no_match_items == 1
assert result.progress.no_match_items == 0
assert result.progress.failed_items == 1
assert len(calls) == 1
@@ -496,7 +500,7 @@ def test_real_failure_isolated_from_later_project(tmp_path):
assert projects[0].status is ProjectStatus.COMPLETED
assert projects[1].status is ProjectStatus.FAILED
assert projects[2].status is ProjectStatus.COMPLETED
assert service.get_task(task.task_id).status is TaskStatus.RUNNING
assert service.get_task(task.task_id).status is TaskStatus.FAILED
assert service.quick_run_phase(task.task_id) == 'retry'
@@ -512,7 +516,7 @@ def test_batch_failure_marks_project_failed_and_continues_to_next_project(tmp_pa
assert projects[0].status is ProjectStatus.FAILED
assert projects[0].summary == 'quick_run_result:0:0:failed|Main:1 result'
assert projects[1].status is ProjectStatus.COMPLETED
assert service.get_task(task.task_id).status is TaskStatus.RUNNING
assert service.get_task(task.task_id).status is TaskStatus.FAILED
assert service.quick_run_phase(task.task_id) == 'retry'
@@ -555,7 +559,7 @@ def test_missing_project_is_skipped_and_other_projects_continue(tmp_path):
assert final_projects[0].status is ProjectStatus.FAILED
assert '工程文件不存在,已跳过' in final_projects[0].summary
assert final_projects[1].status is ProjectStatus.COMPLETED
assert service.get_task(task.task_id).status is TaskStatus.RUNNING
assert service.get_task(task.task_id).status is TaskStatus.FAILED
def test_batch_failure_continues_with_next_rule_in_same_project(tmp_path):
@@ -602,7 +606,7 @@ def test_batch_failure_continues_with_next_rule_in_same_project(tmp_path):
assert result.progress.saved_items == 1
assert result.progress.failed_items == 1
assert service.projects(task.task_id)[0].status is ProjectStatus.FAILED
assert service.get_task(task.task_id).status is TaskStatus.RUNNING
assert service.get_task(task.task_id).status is TaskStatus.FAILED
assert service.quick_run_phase(task.task_id) == 'retry'
status_text = (data / 'quick-run/status.tsv').read_text(encoding='utf-8')
assert 'batch_skipped' not in status_text
+63
View File
@@ -0,0 +1,63 @@
"""Task categories follow committed receipts before the whole batch releases."""
from threading import Event
from PySide6.QtCore import Qt
from dx_productivity.trial_task_center.models import TaskStatus
from dx_productivity.trial_task_center.service import TaskCenterService
from dx_productivity.trial_task_center.batch_resources import BatchResourceCleanupError
def visible_ids(panel):
return [panel.list.item(index).data(Qt.ItemDataRole.UserRole) for index in range(panel.list.count())]
def test_completed_task_moves_category_while_next_task_is_still_executing(ui, qtbot, monkeypatch):
window=ui.window
first,second=ui.tasks
waiting,release=Event(),Event()
execute=TaskCenterService.execute_quick_run_queue
def hold_second(service, task_id, **options):
if task_id==second.task_id:
waiting.set()
assert release.wait(10), 'test did not release its controlled second task'
return execute(service, task_id, **options)
monkeypatch.setattr(TaskCenterService,'execute_quick_run_queue',hold_second)
window._quick_run_confirmer=lambda *_args:True
try:
window._execute_project_tasks()
qtbot.waitUntil(waiting.is_set, timeout=5000)
assert ui.store.get_task(first.task_id).status is TaskStatus.COMPLETED
ui.open(second)
window.task_list.filter.setCurrentIndex(window.task_list.filter.findData('completed'))
qtbot.waitUntil(lambda:visible_ids(window.task_list)==[first.task_id], timeout=3000)
assert window._workers and window._batch_stop is not None
assert window._task_id==second.task_id
assert ui.store.get_task(second.task_id).status is not TaskStatus.COMPLETED
assert len(ui.calls)==1
finally:
release.set()
ui.settle()
assert len(ui.calls)==2
assert set(visible_ids(window.task_list))=={first.task_id,second.task_id}
def test_resource_error_after_business_completion_refreshes_categories(ui):
window=ui.window
first=ui.tasks[0]
window._begin_run(first.task_id)
window._quick_run_timer.stop() # The final error path must refresh even without a timer tick.
window.task_list.filter.setCurrentIndex(window.task_list.filter.findData('completed'))
def execute_then_fail_release():
progress=ui.service.quick_run_progress(first.task_id)
expected=(str(progress.next_project_path),progress.next_old_part_number,
progress.next_new_part_number,'preserve')
ui.service.execute_quick_run_queue(first.task_id,expected_item=expected,
stop_event=window._batch_stop)
raise BatchResourceCleanupError('controlled desktop release failure after task completion')
window._run_async(execute_then_fail_release,lambda _:None,expected_task_id=first.task_id)
ui.settle()
assert ui.store.get_task(first.task_id).status is TaskStatus.COMPLETED
assert visible_ids(window.task_list)==[first.task_id]
assert window._batch_stop is None
assert len(ui.calls)==1
assert 'controlled desktop release failure' in window.run_panel.operator_band.text()
+39 -2
View File
@@ -28,10 +28,14 @@ def visible_ids(panel):
@pytest.mark.parametrize(('category','expected'),[
('all',['draft','ready','running','paused','completed','failed']),
('unfinished',['draft','ready','running','paused','failed']),
('all',['draft','ready','running','paused','completed','failed','blocked']),
('unfinished',['draft','ready','running','paused','failed','blocked']),
('pending',['draft','ready']),
('running',['running']),
('paused',['paused']),
('completed',['completed']),
('failed',['failed']),
('blocked',['blocked']),
])
def test_visible_status_category_filters_task_list(navigation,category,expected):
panel,tasks,opened=navigation
@@ -89,3 +93,36 @@ def test_recycle_bin_ignores_active_status_category(ui):
assert panel.filter.isEnabled()
assert panel.filter.currentData()=='completed'
assert visible_ids(panel)==[]
def test_retryable_failure_is_in_failed_category_and_never_displayed_as_running(navigation):
panel,tasks,opened=navigation
failed = replace(tasks[2], summary='current-sheet quick-run queue completed with retryable failures')
panel.populate(tuple(failed if task.task_id == failed.task_id else task for task in tasks))
panel.filter.setCurrentIndex(panel.filter.findData('failed'))
assert visible_ids(panel)==['running','failed']
assert panel.list.item(0).text().splitlines()[-1]=='失败'
panel.filter.setCurrentIndex(panel.filter.findData('running'))
assert visible_ids(panel)==[]
assert failed.status is TaskStatus.RUNNING # Display classification does not authorize a retry.
def test_status_refresh_preserves_multiple_selected_tasks(navigation):
panel,tasks,opened=navigation
panel.select_all_tasks()
updated=tuple(replace(task,status=TaskStatus.COMPLETED) if task.task_id=='ready' else task for task in tasks)
panel.populate(updated,selected_task_id='ready')
assert {item.data(Qt.ItemDataRole.UserRole) for item in panel.list.selectedItems()} == {
task.task_id for task in tasks}
def test_status_refresh_does_not_reselect_a_current_item_the_user_deselected(navigation):
panel,tasks,opened=navigation
panel.select_all_tasks()
panel.list.currentItem().setSelected(False)
before={item.data(Qt.ItemDataRole.UserRole) for item in panel.list.selectedItems()}
assert 'ready' not in before
updated=tuple(replace(task,status=TaskStatus.COMPLETED) if task.task_id=='ready' else task for task in tasks)
panel.populate(updated,selected_task_id='ready')
assert {item.data(Qt.ItemDataRole.UserRole) for item in panel.list.selectedItems()}==before
assert panel.current_task_id()=='ready'
+7 -1
View File
@@ -25,8 +25,12 @@ def run_vbs(tmp_path, names, body, prelude=''):
names = tuple(names)
support = tuple(name for name in ('QPReadOptionalAttributeValue', 'QPRequiredAttributeValue',
'QPReadableAttributeValue', 'QPEffectiveAttributeValue',
'QPComponentIdentityKey', 'QPComparableState', 'QPEncode', 'QPDecode')
'QPComponentIdentityKey', 'QPComponentUnitKey', 'QPBindSheetReferences',
'QPBindSnapshotReference', 'QPBindSnapshotFullState', 'QPCommandTargetKey',
'QPComparableState', 'QPEncode', 'QPDecode')
if name not in {n.lstrip('?') for n in names})
if not re.search(r'(?mi)^Dim [^\n]*\bQP_ReferenceAliases\b', prelude + '\n' + body):
prelude += '\nDim QP_ReferenceAliases\n'
script.write_text(prelude + '\n' + '\n'.join(routine(name) for name in names + support) + '\n' + body,
encoding='ascii')
result = subprocess.run(['cscript.exe', '//nologo', str(script)], capture_output=True,
@@ -112,6 +116,8 @@ End Sub
Function QPUntouchedFullState(ByRef component)
QPUntouchedFullState = component
End Function
Sub QPProgress(ByVal Stage,ByVal Done,ByVal Total)
End Sub
''')
assert output=='stable semantic state'
+10 -13
View File
@@ -1,4 +1,4 @@
"""An external Windows reference blocks a batch without replaying its results."""
"""An external Windows reference is isolated after owned writers exit."""
import json
import os
from pathlib import Path
@@ -21,26 +21,23 @@ def test_external_reference_preserves_completed_result_until_resource_only_clean
runtime.bind_logs(configure_logging(tmp_path), ('windows-resource-test',))
external = None
try:
with pytest.raises(BatchResourceCleanupError) as caught:
with runtime:
with runtime.native_scope(tmp_path / 'completed-operation') as scope:
with runtime:
with runtime.native_scope(tmp_path / 'completed-operation') as scope:
# The test owns this independent actor, outside runtime's
# process/job list. The runtime must leave it alive.
external = _Win32Api().create_process((sys.executable, '-u', '-c',
external = _Win32Api().create_process((sys.executable, '-u', '-c',
'import win32service,time,sys; '
'd=win32service.OpenDesktop(sys.argv[1],0,False,0x10000000); '
'd.SetThreadDesktop();print("ready",flush=True);time.sleep(30)', scope.name),
build_startup_info(scope.name).lpDesktop, None, None,
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
assert external.stdout.readline() == b'ready\r\n'
runtime.result = 'already saved and independently verified'
assert caught.value.completed_result == runtime.result
assert external.stdout.readline() == b'ready\r\n'
runtime.result = 'already saved and independently verified'
assert external.poll() is None
assert json.loads(runtime.path.read_text())['state'] == 'blocked'
assert json.loads(runtime.path.read_text())['state'] == 'isolated'
log = runtime.path.with_name('resource-release.log').read_text(encoding='utf-8')
assert '资源释放失败' in log and '桌面消失:未确认' in log
with pytest.raises(BatchResourceCleanupError):
guard_pending_resources(tmp_path)
assert 'isolated' in log
guard_pending_resources(tmp_path)
assert not list(Path(tmp_path).rglob('uncertain.flag'))
assert current_desktop_name() == before
finally:
@@ -52,7 +49,7 @@ def test_external_reference_preserves_completed_result_until_resource_only_clean
assert runtime.generation == 1
rows = [json.loads(line) for line in (tmp_path / 'tasks/windows-resource-test/events.jsonl')
.read_text(encoding='utf-8').splitlines()]
assert [row['operation'] for row in rows] == ['resource_release_started', 'resource_release_failed',
assert [row['operation'] for row in rows] == ['resource_release_started', 'resource_release_isolated',
'resource_release_started', 'resource_release_succeeded']
assert rows[-1]['details']['desktop_disappeared'] is True
guard_pending_resources(tmp_path)
+13
View File
@@ -419,3 +419,16 @@ def test_retained_scope_honors_optional_selectors_and_disabled_overrides(batch,
retained = retained_bom_rows((request,), rows)
assert len(retained) == 1 and retained[0].target.uid == retained_uid
assert retained[0] is next(row for row in rows if row.target.uid == retained_uid)
def test_unique_pending_reference_does_not_retain_old_row_due_to_stale_uid(batch):
from dx_productivity.trial_task_center.bom_pending import PendingBomRequest, retained_bom_rows
from dx_productivity.trial_task_center.bom_plan import BomRecord
from dx_productivity.trial_task_center.target_plan import TargetPlanRow
service, store, tasks, targets, calls = batch
first, second = targets[tasks[0].task_id]
rows = (TargetPlanRow(first, True, 'KEEP-1'), TargetPlanRow(second, True, 'KEEP-2'))
record = BomRecord(str(first.project_path), 'OLD', 'NEW', (first.refdes,),
first.schematic_name, first.sheet_number, 'stale-uid', False, '', True)
request = PendingBomRequest(record, 'source.xlsx', 8, 'hash')
assert retained_bom_rows((request,), rows) == (rows[1],)
+20
View File
@@ -20,6 +20,26 @@ def setup_store(tmp_path):
return store, task, targets
def test_unique_scoped_reference_ignores_stale_bom_uid(tmp_path):
from dx_productivity.trial_task_center.bom_plan import BomMatcher, BomRecord
from dx_productivity.trial_task_center.target_plan import TargetPlanRow
target = ComponentTarget(tmp_path / 'board.prj', 'MAIN', '1', 'current', 'R1', 'OLD', '10nF')
matcher = BomMatcher((TargetPlanRow(target),))
record = BomRecord('', 'OLD', 'NEW', ('R1',), 'MAIN', '1', 'stale', True, '', True)
matched = matcher.match_record(record)
assert len(matched) == 1 and next(iter(matched.values())).target is target
def test_uid_can_only_disambiguate_multiple_reference_candidates(tmp_path):
from dx_productivity.trial_task_center.bom_plan import BomMatcher, BomRecord
from dx_productivity.trial_task_center.target_plan import TargetPlanRow
first = ComponentTarget(tmp_path / 'board.prj', 'MAIN', '1', 'u1', 'R1', 'OLD', '10nF')
second = ComponentTarget(tmp_path / 'board.prj', 'MAIN', '2', 'u2', 'R1', 'OLD', '10nF')
matcher = BomMatcher((TargetPlanRow(first), TargetPlanRow(second)))
record = BomRecord('', 'OLD', 'NEW', ('R1',), '', '', 'u2', True, '', True)
assert next(iter(matcher.match_record(record).values())).target is second
def test_already_target_does_not_accept_duplicate_nc_from_leading_marker(tmp_path):
from dx_productivity.trial_task_center.bom_plan import BomMatcher, BomRecord
from dx_productivity.trial_task_center.target_plan import TargetPlanRow
+163
View File
@@ -0,0 +1,163 @@
import json
import os
import zipfile
import pytest
from dx_productivity.trial_task_center import diagnostics_bundle as bundle
from dx_productivity.trial_task_center.execution_feedback import read_feedback
from dx_productivity.trial_task_center.ui.execution_feedback import format_feedback
def write(root, name, text):
path = root / name
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(text, encoding='utf-8')
return path
def test_bundle_exports_execution_closure_and_inventory_evidence(tmp_path):
names = {
'tasks/t/quick-run/worker-progress.json': '{"stage":"save_and_close"}',
'tasks/t/quick-run/quick-part-replace.log': 'native returned token=private-token',
'tasks/t/quick-run/project-commit-' + 'a' * 32 + '.tsv': 'saved_ok\tjob1',
'tasks/t/quick-run/project-pause-' + 'a' * 32 + '.tsv': 'paused\tjob2',
'tasks/t/quick-run/post-state-job1.tsv': 'R1\tNEW',
'tasks/t/quick-run/value-readback-job1.json': '{"verified":true}',
'execution-proof/proof1/project-commit-' + 'a' * 32 + '.tsv': 'saved_ok\tjob1',
'execution-proof/proof1/completion.json': '{"verified":true}',
'tasks/t/quick-run-history/native-inventory-a/native-inventory.json': '{"state":"ready"}',
'tasks/t/quick-run-history/native-inventory-a/output/scope.txt': 'R1\tUID1',
'tasks/t/quick-run-history/native-inventory-a/output/effective.txt': 'R1\tNEW',
'tasks/t/netlist-checks/run1/verification.json': '{"verified":true}',
'tasks/t/netlist-checks/run1/after-export/netlist.json': '{"complete":true}',
}
for name, text in names.items():
write(tmp_path, name, text)
write(tmp_path, 'tasks/t/netlist-checks/run1/before/project.prj', 'private project')
output = bundle.export_diagnostic_bundle(tmp_path, tmp_path / 'support.zip')
with zipfile.ZipFile(output) as archive:
assert set(names).issubset(archive.namelist())
assert 'private-token' not in archive.read('tasks/t/quick-run/quick-part-replace.log').decode()
assert all(not name.endswith('.prj') for name in archive.namelist())
def test_bundle_tails_active_logs_and_keeps_latest_evidence_under_budget(tmp_path, monkeypatch):
monkeypatch.setattr(bundle, 'MAX_FILE_BYTES', 160)
monkeypatch.setattr(bundle, 'MAX_FILES', 2)
for index in range(4):
path = write(tmp_path, f'tasks/{index}/quick-run/worker-progress.json', json.dumps({'stage': str(index)}))
os.utime(path, (100 + index, 100 + index))
log = write(tmp_path, 'tasks/latest/quick-run/quick-part-replace.log',
'old heartbeat\n' * 100 + 'last failure token=secret-value\n')
os.utime(log, (500, 500))
output = bundle.export_diagnostic_bundle(tmp_path, tmp_path / 'support.zip')
with zipfile.ZipFile(output) as archive:
text = archive.read('tasks/latest/quick-run/quick-part-replace.log').decode()
assert 'last failure' in text and 'secret-value' not in text
assert 'tasks/3/quick-run/worker-progress.json' in archive.namelist()
manifest = json.loads(archive.read('manifest.json'))
assert manifest['truncated'][0]['mode'] == 'tail'
@pytest.mark.parametrize('record,reason,wait', [
({'stage': 'native_replace', 'stop_requested': True}, 'inactivity', 'native_call'),
({'stage': 'save_and_close', 'stopping_reason': 'user_pause'}, 'user_pause', 'save'),
({'stage': 'verify_saved_project'}, '', 'readback'),
({'stage': 'private_shutdown', 'stopping_reason': 'failure_cleanup', 'wait_reason': 'mentor_exit'},
'failure_cleanup', 'mentor_exit'),
({'stage': 'awaiting_reconciliation', 'primary_error': 'native failed'}, '', 'reconciliation'),
])
def test_feedback_classifies_stop_and_actual_wait_without_generic_save(tmp_path, record, reason, wait):
write(tmp_path, 'quick-run/worker-progress.json', json.dumps(record))
value = read_feedback('t', 1, 1, 'running', tmp_path, 0)
assert value.stopping_reason == reason
assert value.wait_reason == wait
text = format_feedback(value, 'board', 4, True)
if wait != 'save':
assert '保存' not in text
if record.get('primary_error'):
assert record['primary_error'] in text
def test_operation_events_preserve_last_success_and_incomplete_call(tmp_path):
from dx_productivity.trial_task_center.operation_diagnostics import record_operation
assert record_operation(tmp_path / 'quick-run', 'item', 'return', item_id='job1')
assert record_operation(tmp_path / 'quick-run', 'save', 'enter', item_id='job2')
assert record_operation(tmp_path / 'quick-run', 'mentor_exit', 'enter')
assert record_operation(tmp_path / 'quick-run', 'mentor_exit', 'error', error='cleanup failed')
output = bundle.export_diagnostic_bundle(tmp_path, tmp_path / 'support.zip')
with zipfile.ZipFile(output) as archive:
summary = json.loads(archive.read('closure-summary.json'))['operations']['quick-run']
assert summary['last_successful_item'] == 'job1'
assert summary['incomplete_operations'][0]['operation'] == 'save'
assert summary['errors'][0]['error'] == 'cleanup failed'
def test_stop_control_keeps_failure_reason_and_first_error(tmp_path):
from dx_productivity.trial_task_center.stop_control import StopControl, wait_for_resolution, MentorReleaseRequired
control = StopControl()
with pytest.raises(MentorReleaseRequired):
wait_for_resolution(control, tmp_path, 'native failed', release_on_failure=True)
assert control.stopping_reason == 'failure_cleanup'
assert control.primary_error == 'native failed'
control.request_failure_cleanup(tmp_path, 'later cleanup failure')
assert control.primary_error == 'native failed'
record = json.loads((tmp_path / 'worker-progress.json').read_text())
assert record['primary_error'] == 'native failed'
assert record['wait_reason'] == 'reconciliation'
def test_closing_desktop_overrides_old_save_wait(tmp_path):
write(tmp_path, 'quick-run/worker-progress.json', json.dumps({
'stage': 'save_and_close', 'wait_reason': 'save', 'stopping_reason': 'failure_cleanup'}))
write(tmp_path, 'quick-run/private-desktop.json', '{"state":"closing"}')
value = read_feedback('t', 1, 1, 'running', tmp_path, 0)
assert value.wait_reason == 'desktop_release'
assert '保存' not in format_feedback(value, 'board', 4, True)
def test_bundle_keeps_complete_event_tail_and_latest_successful_status(tmp_path):
write(tmp_path, 'quick-run/operation-events.jsonl',
'{"operation":"save","event":"enter","item_id":"job2"}\n{"broken":')
write(tmp_path, 'quick-run/status.tsv',
'job1\tsaved_ok\tsession_verified\tdone\njob2\tpending_save\tsession_staged\tstaged\n')
output = bundle.export_diagnostic_bundle(tmp_path, tmp_path / 'support.zip')
with zipfile.ZipFile(output) as archive:
summary = json.loads(archive.read('closure-summary.json'))
assert summary['items']['quick-run']['last_successful_item'] == 'job1'
assert summary['items']['quick-run']['latest_item'] == {'job_id': 'job2', 'state': 'pending_save'}
assert summary['operations']['quick-run']['incomplete_operations'][0]['operation'] == 'save'
manifest = json.loads(archive.read('manifest.json'))
assert any(row['reason'] == 'invalid_jsonl_lines' for row in manifest['omissions'])
def test_bundle_keeps_valid_call_when_other_event_has_invalid_shape(tmp_path):
write(tmp_path, 'quick-run/operation-events.jsonl',
'{"operation":[],"event":"enter","item_id":{}}\n'
'{"operation":"save","event":"enter","item_id":"job1"}\n')
output = bundle.export_diagnostic_bundle(tmp_path, tmp_path / 'support.zip')
with zipfile.ZipFile(output) as archive:
summary = json.loads(archive.read('closure-summary.json'))
assert summary['operations']['quick-run']['incomplete_operations'][0]['operation'] == 'save'
def test_bundle_reads_native_ascii_report_using_local_windows_encoding(tmp_path):
name = 'quick-run-history/native-inventory-a/output/scope.txt'
path = write(tmp_path, name, '')
path.write_bytes('R1 中文说明'.encode('mbcs' if os.name == 'nt' else 'cp1252', errors='replace'))
output = bundle.export_diagnostic_bundle(tmp_path, tmp_path / 'support.zip')
with zipfile.ZipFile(output) as archive:
assert name in archive.namelist()
assert 'R1 ' in archive.read(name).decode('utf-8')
def test_redaction_growth_does_not_discard_latest_failure_tail(tmp_path, monkeypatch):
monkeypatch.setattr(bundle, 'MAX_FILE_BYTES', 128)
name = 'quick-run/quick-part-replace.log'
write(tmp_path, name, 'token=x\n' * 100 + 'native failed token=y\n')
output = bundle.export_diagnostic_bundle(tmp_path, tmp_path / 'support.zip')
with zipfile.ZipFile(output) as archive:
text = archive.read(name).decode()
assert 'native failed' in text
assert len(archive.read(name)) <= 128
+161
View File
@@ -0,0 +1,161 @@
import json
from pathlib import Path
import subprocess
import sys
import pytest
from dx_productivity.trial_task_center import batch_resources, batch_worker, operation_diagnostics
from dx_productivity.trial_task_center.project_session import ProjectSessionWorker
from dx_productivity.trial_task_center.worker_monitor import WorkerMonitorError
from test_project_session import toy_worker
def events(root):
path = root / 'operation-events.jsonl'
return [json.loads(line) for line in path.read_text(encoding='utf-8').splitlines()] if path.exists() else []
def test_native_step_and_commit_enter_events_precede_real_worker_dispatch(tmp_path):
command = toy_worker(tmp_path)
script = Path(command[2])
source = script.read_text(encoding='utf-8').replace('job=initial; sequence=0', '''job=initial; sequence=0
events=[json.loads(line) for line in (root/'operation-events.jsonl').read_text().splitlines()]
assert any(e['operation']=='native_step' and e['event']=='enter' and e['item_id']==job for e in events)''')
source = source.replace("if job=='close':break", '''if job=='close':
events=[json.loads(line) for line in (root/'operation-events.jsonl').read_text().splitlines()]
assert any(e['operation']=='close_project' and e['event']=='enter' for e in events)
break''')
script.write_text(source, encoding='utf-8')
worker = ProjectSessionWorker(tmp_path / 'board.prj', tmp_path)
try:
assert worker.run_step(command, 3).returncode == 0
assert [(e['operation'], e['event']) for e in events(tmp_path)] == [('native_step', 'enter'), ('native_step', 'return')]
assert not (tmp_path / 'closed.txt').exists()
finally:
worker.close()
assert [(e['operation'], e['event']) for e in events(tmp_path)] == [
('native_step', 'enter'), ('native_step', 'return'), ('close_project', 'enter'), ('close_project', 'return')]
assert not any(e['operation'] in {'item', 'readback'} for e in events(tmp_path))
@pytest.mark.parametrize('diagnostic_failure', [False, True])
def test_native_failure_keeps_primary_error_even_when_diagnostics_fail(tmp_path, monkeypatch, diagnostic_failure):
command = toy_worker(tmp_path)
script = Path(command[2])
script.write_text(script.read_text(encoding='utf-8').replace('saved_ok', 'invalid_checkpoint'), encoding='utf-8')
if diagnostic_failure:
monkeypatch.setattr(operation_diagnostics, 'record_operation', lambda *args, **kwargs: (_ for _ in ()).throw(OSError('diagnostic disk failure')))
worker = ProjectSessionWorker(tmp_path / 'board.prj', tmp_path)
try:
with pytest.raises(WorkerMonitorError, match='invalid checkpoint'):
worker.run_step(command, 3)
finally:
worker.close()
if not diagnostic_failure:
records = [row for row in events(tmp_path) if row['operation'] == 'native_step']
assert [row['event'] for row in records] == ['enter', 'error']
assert 'invalid checkpoint' in records[-1]['error']
def test_diagnostic_failure_does_not_block_worker_success_or_native_close(tmp_path, monkeypatch):
monkeypatch.setattr(operation_diagnostics, 'record_operation', lambda *args, **kwargs: (_ for _ in ()).throw(OSError('diagnostic disk failure')))
worker = ProjectSessionWorker(tmp_path / 'board.prj', tmp_path)
try:
assert worker.run_step(toy_worker(tmp_path), 3).returncode == 0
finally:
worker.close()
assert (tmp_path / 'closed.txt').exists()
def test_native_close_failure_logs_error_and_never_a_successful_close(tmp_path):
command = toy_worker(tmp_path)
script = Path(command[2])
script.write_text(script.read_text(encoding='utf-8') + '\nsys.exit(7)\n', encoding='utf-8')
worker = ProjectSessionWorker(tmp_path / 'board.prj', tmp_path)
assert worker.run_step(command, 3).returncode == 0
with pytest.raises(WorkerMonitorError, match='did not close cleanly'):
worker.close()
closing = [row for row in events(tmp_path) if row['operation'] == 'close_project']
assert [row['event'] for row in closing] == ['enter', 'error']
worker.close()
assert [row for row in events(tmp_path) if row['operation'] == 'close_project'] == closing
def test_result_persistence_enter_is_durable_before_actual_result_write(tmp_path, monkeypatch):
original = batch_resources.write_record
def observe(path, value):
assert [(e['operation'], e['event']) for e in events(tmp_path)] == [('result_persistence', 'enter')]
assert not path.exists()
original(path, value)
monkeypatch.setattr(batch_resources, 'write_record', observe)
result = {'attempt_id': 'attempt', 'payload': {'status': 'ok'}}
batch_worker._persist_result(tmp_path, result)
assert json.loads((tmp_path / 'result.json').read_text(encoding='utf-8')) == result
assert [(e['operation'], e['event']) for e in events(tmp_path)] == [('result_persistence', 'enter'), ('result_persistence', 'return')]
@pytest.mark.parametrize('diagnostic_failure', [False, True])
def test_result_write_failure_remains_original_exception_and_never_logs_success(tmp_path, monkeypatch, diagnostic_failure):
failure = OSError('result storage failed')
monkeypatch.setattr(batch_resources, 'write_record', lambda *args: (_ for _ in ()).throw(failure))
if diagnostic_failure:
monkeypatch.setattr(operation_diagnostics, 'record_operation', lambda *args, **kwargs: (_ for _ in ()).throw(OSError('diagnostic disk failure')))
with pytest.raises(OSError) as caught:
batch_worker._persist_result(tmp_path, {'attempt_id': 'attempt'})
assert caught.value is failure
assert not (tmp_path / 'result.json').exists()
if not diagnostic_failure:
assert [e['event'] for e in events(tmp_path)] == ['enter', 'error']
assert events(tmp_path)[-1]['error'] == 'result storage failed'
def test_diagnostic_failure_does_not_block_actual_result_persistence(tmp_path, monkeypatch):
monkeypatch.setattr(operation_diagnostics, 'record_operation', lambda *args, **kwargs: (_ for _ in ()).throw(OSError('diagnostic disk failure')))
batch_worker._persist_result(tmp_path, {'attempt_id': 'attempt', 'payload': 'completed'})
assert json.loads((tmp_path / 'result.json').read_text(encoding='utf-8'))['payload'] == 'completed'
def test_real_supervisor_result_write_uses_structured_persistence_boundary(tmp_path):
from dx_productivity.trial_task_center.batch_supervisor import run_supervised
from dx_productivity.trial_task_center.logging_setup import configure_logging
from dx_productivity.trial_task_center.service import TaskCenterService
from dx_productivity.trial_task_center.store import TaskStore
store = TaskStore(tmp_path / 'tasks.db')
try:
service = TaskCenterService(store, configure_logging(tmp_path))
result = run_supervised(service, 'probe', {})
assert result['resource_state'] == 'not_acquired'
directory = next((tmp_path / 'batch-resources').iterdir())
assert [row['event'] for row in events(directory) if row['operation'] == 'result_persistence'] == ['enter', 'return']
assert (directory / 'result.json').exists()
finally:
store.close()
def test_result_write_process_crash_leaves_durable_incomplete_boundary(tmp_path):
source = Path(__file__).resolve().parents[2] / 'src'
script = '''import os,sys
from pathlib import Path
sys.path.insert(0,sys.argv[1])
from dx_productivity.trial_task_center import batch_resources,batch_worker
batch_resources.write_record=lambda *args: os._exit(23)
batch_worker._persist_result(Path(sys.argv[2]),{'attempt_id':'attempt'})
'''
result = subprocess.run((sys.executable, '-c', script, str(source), str(tmp_path)), capture_output=True, timeout=10)
assert result.returncode == 23, result.stderr
summary = operation_diagnostics.summarize_operations(events(tmp_path))
assert [(e['operation'], e['item_id']) for e in summary['incomplete_operations']] == [('result_persistence', 'attempt')]
assert not (tmp_path / 'result.json').exists()
def test_incomplete_actual_operation_can_be_reported_before_return(tmp_path):
with operation_diagnostics.operation_span(tmp_path, 'native_step', item_id='job'):
summary = operation_diagnostics.summarize_operations(events(tmp_path))
assert [(e['operation'], e['item_id']) for e in summary['incomplete_operations']] == [('native_step', 'job')]
assert summary['last_successful_item'] is None
assert operation_diagnostics.summarize_operations(events(tmp_path))['incomplete_operations'] == []
def test_diagnostic_serialization_failure_is_nonblocking(tmp_path):
assert operation_diagnostics.record_operation(tmp_path, 'native_step', 'enter', item_id=object()) is False
@@ -116,17 +116,28 @@ def test_manual_failure_cannot_be_completed_by_native_progress(failed_native):
service.quick_run_progress(task_id)
def test_uncertain_batch_failed_blocks_next_project_in_task_queue(failed_native, tmp_path):
def test_uncertain_batch_failed_isolated_before_next_project_in_task_queue(failed_native, tmp_path, monkeypatch):
from threading import Event
from dx_productivity.trial_task_center.task_queue import execute_task_queue
service, store, task_id, _, _ = failed_native
other = store.create_task('next must not run')
other = store.create_task('independent next project')
path = tmp_path / 'next.prj'
path.touch()
project = store.sync_projects(other.task_id, tmp_path, (path,))[0]
store.set_project_selected(project.project_id, True)
store.bind_project_task(other.task_id, project.project_id, 'batch', 1)
result = execute_task_queue(service, (task_id, other.task_id), stop_event=Event())
assert result.paused
assert [entry.state for entry in result.entries] == ['blocked', 'pending']
from test_project_isolation_queue import ReleasedRuntime
from dx_productivity.trial_task_center.quick_run_protocol import QuickRunProgress
calls = []
child = service.for_project_task(other.task_id)
monkeypatch.setattr(child, 'quick_run_progress', lambda _: calls.append(other.task_id) or QuickRunProgress(
0, 0, None, None, None, (path,), True))
monkeypatch.setattr(child, 'has_interrupted_quick_run', lambda _: False)
child._quick_run_preparer.reconcile_terminal_receipts = lambda _: None
original_child = service.for_project_task
monkeypatch.setattr(service, 'for_project_task', lambda identifier: child if identifier == other.task_id else original_child(identifier))
result = execute_task_queue(service, (task_id, other.task_id), stop_event=Event(), runtime=ReleasedRuntime(service._logging.root))
assert not result.paused
assert [entry.state for entry in result.entries] == ['blocked', 'completed']
assert calls == [other.task_id]
assert not (service.for_project_task(other.task_id)._quick_run_preparer.data_root / 'quick-run').exists()
+5
View File
@@ -42,6 +42,11 @@ def test_inventory_rejects_changed_or_removed_baseline_atomically(batch, change)
incoming.append(incoming[0])
else:
incoming[0] = replace(incoming[0], **{change: 'CHANGED'})
if change == 'uid':
rows = store.record_target_inventory(task.task_id, incoming)
assert len(rows) == 3
assert {row.target.key: row for row in rows}[before[0].target.key] == before[0]
return
with pytest.raises(TaskCenterError):
store.record_target_inventory(task.task_id, incoming)
assert store.list_target_plan(task.task_id) == before
@@ -0,0 +1,230 @@
from pathlib import Path
from types import SimpleNamespace
import pytest
from test_failed_native_recovery_guard import failed_native
from dx_productivity.trial_task_center import batch_resources, batch_runtime
from dx_productivity.trial_task_center.stop_control import StopControl, ProjectStopControl
def test_project_force_observes_user_force_without_leaking_failure_to_parent(tmp_path):
from dx_productivity.trial_task_center.workspace_execution import _ProjectControl
parent = StopControl()
local = _ProjectControl(parent)
local.request_failure_cleanup(tmp_path, 'A failure')
assert local.force_event.is_set()
assert not parent.force_event.is_set() and not parent.is_set()
successor = _ProjectControl(parent)
assert not successor.force_event.is_set() and not successor.is_set()
parent.request_force()
assert successor.is_set() and successor.force_event.wait(.01)
@pytest.mark.parametrize('residual', [False, True])
def test_failed_project_retires_proven_exit_despite_shutdown_error(tmp_path, residual):
from test_batch_runtime import Desktop
class OwnedDesktop(Desktop):
def close(self):
self.cleanup_record = {'desktop_handle_closed': True, 'owned_cleanup_complete': True,
'desktop_confirmation': {'released': not residual}, 'errors': []}
if residual: raise OSError('foreign desktop reference')
runtime = batch_runtime.BatchCadRuntime(tmp_path, 'execution', ProjectStopControl(StopControl()), desktop_factory=OwnedDesktop)
runtime._ensure_desktop()
def cad_exit(*args): raise OSError('shutdown diagnostics failed after exit')
runtime.cad = SimpleNamespace(cad=SimpleNamespace(poll=lambda: 0), record={}, __exit__=cad_exit)
original = RuntimeError('original failed project')
assert runtime.release_failed_project(original, tmp_path / 'failed')
assert batch_resources.blocking_resources(tmp_path) == ()
assert runtime.record['state'] == ('isolated' if residual else 'released')
assert runtime.record['release_results']['desktop_released'] is (not residual)
assert runtime.desktop is None and 'shutdown diagnostics failed' in original.shutdown_warning
assert (tmp_path / 'failed' / 'uncertain.flag').exists()
def test_runtime_cleanup_events_record_real_boundaries_and_preserve_error(tmp_path):
import json
from test_batch_runtime import Desktop
runtime = batch_runtime.BatchCadRuntime(tmp_path, 'execution', desktop_factory=Desktop)
runtime._ensure_desktop()
def cad_exit(*args): raise OSError('actual exit error')
runtime.cad = SimpleNamespace(cad=SimpleNamespace(poll=lambda: 0), record={}, __exit__=cad_exit)
with pytest.raises(batch_resources.BatchResourceCleanupError, match='actual exit error'):
runtime.close()
rows = [json.loads(line) for line in (runtime.path.parent / 'operation-events.jsonl').read_text().splitlines()]
assert [(row['operation'], row['event']) for row in rows] == [
('mentor_exit', 'enter'), ('mentor_exit', 'error'), ('desktop_release', 'enter'), ('desktop_release', 'return')]
def test_workspace_accepts_writer_exit_warning_and_blocks_shared_source_follower(tmp_path, monkeypatch):
from test_workspace_execution import service_for
from dx_productivity.trial_task_center.workspace_execution import execute_workspace_projects
from dx_productivity.trial_task_center.errors import TaskCenterError
failure = TaskCenterError('quick_run_uncertain', 'A business outcome unknown')
failure.resources_released = False
failure.owned_writers_exited = True
service, calls = service_for(tmp_path, monkeypatch, {'A': failure, 'B': 'completed'})
result = execute_workspace_projects(service, ('A', 'B'), stop_event=StopControl())
assert not result.paused and result.entries[1].state == 'completed'
calls.clear()
for name in ('A', 'B'):
path = tmp_path / name / 'board.prj'
path.parent.mkdir(exist_ok=True)
path.write_text('KEY iCDBDir "../shared"\n')
result = execute_workspace_projects(service, ('A', 'B'), stop_event=StopControl())
assert not result.paused and [row.state for row in result.entries] == ['blocked', 'blocked']
assert [call[1]['task_ids'][0] for call in calls] == ['A']
def test_blocked_progress_read_does_not_promote_or_throw(failed_native):
from dx_productivity.trial_task_center.models import TaskStatus
service, store, task_id, handoff, _ = failed_native
store.set_task_status(task_id, TaskStatus.BLOCKED, summary='requires reconciliation')
service.quick_run_progress(task_id)
assert store.get_task(task_id).status is TaskStatus.BLOCKED
def test_public_recovery_tries_verified_project_commit_before_cad(failed_native, monkeypatch):
from dx_productivity.trial_task_center.models import TaskStatus
from dx_productivity.trial_task_center.quick_run_protocol import QuickRunStepResult
from dx_productivity.trial_task_center.service import TaskCenterService
service, store, task_id, handoff, _ = failed_native
store.set_task_status(task_id, TaskStatus.BLOCKED, summary='project commit publication failed')
child = service.for_project_task(task_id)
result = QuickRunStepResult(handoff.script_path, handoff.command_path, handoff.status_path,
handoff.log_path, 'saved_ok', 'verified durable project commit')
calls = []
monkeypatch.setattr(child._quick_run_preparer, 'recover_project_commit',
lambda identifier, *, confirmed: calls.append((identifier, confirmed)) or result, raising=False)
monkeypatch.setattr(service, '_synchronize_quick_run_progress', lambda identifier: calls.append(('sync', identifier)))
monkeypatch.setattr(TaskCenterService, '_owned_cad_operation', lambda *args, **options: pytest.fail('project commit recovery acquired CAD'))
assert service.recover_quick_run(task_id, confirmed=True) is result
assert calls == [(task_id, True), ('sync', task_id)]
assert store.get_task(task_id).status is TaskStatus.READY
@pytest.mark.parametrize('publication_failure', [None, 'sql', 'marker'])
@pytest.mark.parametrize('queue_directory', ['task-queues', 'workspace-queues'])
def test_dependency_recovery_needs_confirmation_clear_source_and_never_opens_cad(tmp_path, monkeypatch, publication_failure, queue_directory):
import json
from dx_productivity.trial_task_center.service import TaskCenterService
from dx_productivity.trial_task_center.store import TaskStore
from dx_productivity.trial_task_center.models import TaskStatus
from dx_productivity.trial_task_center.logging_setup import configure_logging
from dx_productivity.trial_task_center.task_queue import _plan_hash
from dx_productivity.trial_task_center.errors import TaskCenterError
store = TaskStore(tmp_path / 'tasks.db')
service = TaskCenterService(store, configure_logging(tmp_path / 'data'))
tasks = []
for name in ('A', 'B'):
task = store.create_task(name)
path = tmp_path / name / 'board.prj'
path.parent.mkdir()
path.write_text(' key iCDBDir "../shared"\n')
project = store.sync_projects(task.task_id, path.parent, (path,))[0]
store.set_project_selected(project.project_id, True)
store.bind_project_task(task.task_id, project.project_id, 'group', len(tasks))
store.add_rule(task.task_id, 'OLD', 'NEW')
tasks.append((task.task_id, path))
for task_id, _ in tasks:
store.set_task_status(task_id, TaskStatus.BLOCKED, summary='dependency')
dependency, follower = tasks
child = service.for_project_task(follower[0])
marker = service._logging.root / 'dependency-blocks' / (follower[0] + '.json')
marker.parent.mkdir()
marker.write_text(json.dumps({'version': 1, 'task_id': follower[0], 'dependency_task_id': dependency[0],
'project_path': str(follower[1].resolve()).casefold(), 'sources': [str((tmp_path / 'shared/icdb.dat').resolve()).casefold()],
'plan_hash': _plan_hash(child, follower[0]), 'queue_id': 'a'*32, 'state': 'blocked', 'undispatched': True}))
queue_path = service._logging.root / queue_directory / ('a'*32 + '.json')
queue_path.parent.mkdir()
queue_path.write_text(json.dumps({'plan_hashes': {follower[0]: _plan_hash(child, follower[0])},
'dependency_tasks': {follower[0]: dependency[0]}, 'entries': {follower[0]: {'state': 'blocked'}}}))
monkeypatch.setattr(TaskCenterService, '_owned_cad_operation', lambda *args, **kw: pytest.fail('dependency recovery opened CAD'))
with pytest.raises(TaskCenterError, match='quick_run_dependency'):
service.recover_quick_run(follower[0], confirmed=False)
with pytest.raises(TaskCenterError, match='quick_run_dependency'):
service.recover_quick_run(follower[0], confirmed=True)
store.set_task_status(dependency[0], TaskStatus.READY, summary='verified recovery fixture')
if publication_failure:
from dx_productivity.trial_task_center import batch_resources
original_write, original_status = batch_resources.write_record, store.set_task_status
failed = []
def write(path, record):
if publication_failure == 'marker' and Path(path) == marker and record.get('state') == 'readmitted' and not failed:
failed.append(True)
raise OSError('readmission marker publication failed')
return original_write(path, record)
def set_status(identifier, state, **options):
if publication_failure == 'sql' and identifier == follower[0] and state is TaskStatus.READY and not failed:
failed.append(True)
raise OSError('SQL readmission failed')
return original_status(identifier, state, **options)
monkeypatch.setattr(batch_resources, 'write_record', write)
monkeypatch.setattr(store, 'set_task_status', set_status)
with pytest.raises(TaskCenterError): service.recover_quick_run(follower[0], confirmed=True)
assert json.loads(marker.read_text())['state'] == 'readmission_prepared'
assert store.get_task(follower[0]).status is (TaskStatus.READY if publication_failure == 'marker' else TaskStatus.BLOCKED)
result = service.recover_quick_run(follower[0], confirmed=True)
assert result.state == 'pending' and store.get_task(follower[0]).status is TaskStatus.READY
assert json.loads(marker.read_text())['state'] == 'readmitted'
assert service.recover_quick_run(follower[0], confirmed=True).state == 'pending'
# A restart sees the same marker/SQL result; it does not need a snapshot.
second_service = TaskCenterService(store, configure_logging(tmp_path / 'data'))
assert second_service.recover_quick_run(follower[0], confirmed=True).state == 'pending'
# The original isolated task stays skipped while its undispatched follower
# can re-enter the same frozen queue after explicit readmission.
from dx_productivity.trial_task_center import task_queue
original_sources = set(json.loads(marker.read_text())['sources'])
queue = {'version': 1, 'task_ids': [dependency[0], follower[0]],
'plan_hashes': {key: _plan_hash(service.for_project_task(key), key) for key, _ in tasks},
'phase': 'finished', 'entries': {dependency[0]: {'state': 'blocked', 'message': 'original uncertain'},
follower[0]: {'state': 'blocked', 'message': 'dependency'}},
'dependency_tasks': {follower[0]: dependency[0]},
'isolated_tasks': {dependency[0]: 'original uncertain', follower[0]: 'dependency'},
'isolated_sources': {source: dependency[0] for source in original_sources}}
queue_path.write_text(json.dumps(queue))
calls = []
from test_project_isolation_queue import ReleasedRuntime
original_child = service.for_project_task
def selected_child(identifier):
selected = original_child(identifier)
if identifier == follower[0]:
selected.has_interrupted_quick_run = lambda _: False
selected._quick_run_preparer.reconcile_terminal_receipts = lambda _: None
from dx_productivity.trial_task_center.quick_run_protocol import QuickRunProgress
selected.quick_run_progress = lambda _: calls.append(identifier) or QuickRunProgress(0, 0, None, None, None, (follower[1],), True)
return selected
monkeypatch.setattr(service, 'for_project_task', selected_child)
def run_queue():
if queue_directory == 'task-queues':
return task_queue.execute_task_queue(service, tuple(key for key, _ in tasks), stop_event=StopControl(),
queue_id='a'*32, runtime=ReleasedRuntime(service._logging.root))
from dx_productivity.trial_task_center.workspace_execution import execute_workspace_projects
return execute_workspace_projects(service, tuple(key for key, _ in tasks), stop_event=StopControl(), queue_id='a'*32)
if queue_directory == 'workspace-queues':
for index, (identifier, _) in enumerate(tasks):
queue['entries'][identifier].update(released=True, problems_done=True, checkpoint_id=str(index+1)*32)
queue_path.write_text(json.dumps(queue))
def supervise(operation, arguments, **controls):
identifier = arguments['task_ids'][0]
calls.append(identifier)
return task_queue.TaskQueueResult(arguments['checkpoint_id'] or arguments['queue_id'],
(task_queue.TaskQueueEntry(identifier, 'completed'),), False)
monkeypatch.setattr(service, '_supervise', supervise)
resumed = run_queue()
assert [row.state for row in resumed.entries] == ['blocked', 'completed'] and calls == [follower[0]]
calls.clear()
run_queue()
assert not calls
attempted_row = {'task_id': follower[0], 'job_id': 'actually-dispatched', 'project_path': str(follower[1]),
'old_part': 'OLD', 'new_part': 'NEW', 'schematic_name': 'MAIN', 'sheet_number': '1'}
child._quick_run_preparer.journal.bind(follower[0], [attempted_row], {})
child._quick_run_preparer.journal.claim(follower[0], attempted_row)
from dx_productivity.trial_task_center import recovery
def native_recovery(*args, **options):
raise TaskCenterError('native_recovery_reached', 'validate the actual follower attempt')
monkeypatch.setattr(recovery, 'recover', native_recovery)
monkeypatch.setattr(TaskCenterService, '_owned_cad_operation', lambda self, name, action, *args, **options: action(SimpleNamespace()))
with pytest.raises(TaskCenterError, match='native_recovery_reached'):
service.recover_quick_run(follower[0], confirmed=True)
store.close()
+5 -2
View File
@@ -17,10 +17,13 @@ def test_explicit_no_button_closes_owned_unknown_popup_and_releases(tmp_path):
process = desktop.start_process((sys.executable, '-c', code), stdout=subprocess.PIPE, stderr=subprocess.PIPE)
guard = CadDialogGuard(desktop, process, tmp_path / 'cad-dialogs.json', 'manual-native-sample')
deadline = time.monotonic() + 10
while not guard.summary()['blocked'] and time.monotonic() < deadline:
dialogs = []
while time.monotonic() < deadline:
guard.poll('manual_sample')
dialogs = guard.summary()['current_dialogs']
if len(dialogs) == 1 and any(button['id'] == 7 for button in dialogs[0]['buttons']):
break
time.sleep(.03)
dialogs = guard.summary()['current_dialogs']
assert len(dialogs) == 1 and process.poll() is None
dialog = dialogs[0]
button = next(button for button in dialog['buttons'] if button['id'] == 7)
+3 -3
View File
@@ -10,7 +10,7 @@ import pytest
@pytest.mark.parametrize('behavior, steps, changed, states', [
('success', 24, 24, ['completed', 'completed', 'completed']),
('failure_continue', 24, 23, ['failed', 'completed', 'completed']),
('uncertain_stop', 1, 0, ['blocked', 'pending', 'pending']),
('uncertain_stop', 17, 16, ['blocked', 'completed', 'completed']),
('pause_resume', 24, 24, ['completed', 'completed', 'completed']),
])
def test_multiple_projects_preserve_task_order_scope_results_and_measurements(tmp_path, behavior, steps, changed, states):
@@ -24,14 +24,14 @@ def test_multiple_projects_preserve_task_order_scope_results_and_measurements(tm
assert report['executed_steps'] == steps
assert report['changed_components'] == changed
assert report['unselected_components_preserved'] == 12
assert report['result']['paused'] == (behavior == 'uncertain_stop')
assert report['result']['paused'] is False
assert [entry['state'] for entry in report['result']['entries']] == states
calls = report['calls']
assert len(calls) == steps
assert all(len(call['refdes']) == 1 for call in calls)
assert len({(call['project'], call['sheet'], call['refdes'][0]) for call in calls}) == steps
assert sum(call['state'] == 'saved_ok' for call in calls) == changed
assert [call['project'] for call in calls] == (['board-000'] if behavior == 'uncertain_stop' else
assert [call['project'] for call in calls] == (['board-000'] + ['board-001'] * 8 + ['board-002'] * 8 if behavior == 'uncertain_stop' else
['board-000'] * 8 + ['board-001'] * 8 + ['board-002'] * 8)
assert calls[0]['state'] == {'failure_continue': 'batch_failed', 'uncertain_stop': 'batch_saved_unverified'}.get(behavior, 'saved_ok')
with sqlite3.connect(output / 'tasks.db') as connection:
+51 -12
View File
@@ -16,7 +16,7 @@ def effective_report(tmp_path, *, part='NEW', value='0R/NC', path='', duplicate=
@pytest.mark.parametrize('repeated_uid', [False, True])
def test_hierarchical_project_can_select_only_globally_unique_root_target(tmp_path, repeated_uid):
def test_hierarchical_project_selects_unambiguous_root_and_child_references(tmp_path, repeated_uid):
hierarchy = report(tmp_path)
text = hierarchy.read_text().replace('Block disconnected \tRoot', 'Block child ')
text = text.replace('Block board \tRoot\n', 'Block board \tRoot\n'
@@ -31,17 +31,13 @@ def test_hierarchical_project_can_select_only_globally_unique_root_target(tmp_pa
text = text.replace('$28I200', '$27I100')
hierarchy.write_text(hierarchy.read_text().replace('$28I200', '$27I100'))
flat.write_text(text)
if repeated_uid:
with pytest.raises(protocol.NativeInventoryUnsupported, match='唯一'):
protocol.read_ascii_discovery(tmp_path / 'board.prj', hierarchy, ('NEW',),
expected_root='board', effective_report_path=flat)
else:
result = protocol.read_ascii_discovery(tmp_path / 'board.prj', hierarchy, ('NEW',),
expected_root='board', effective_report_path=flat)
assert [(row.uid, row.refdes) for row in result.components] == [('$27I100', 'R1')]
with pytest.raises(protocol.NativeInventoryUnsupported):
protocol.read_ascii_discovery(tmp_path / 'board.prj', hierarchy, ('CHILD',),
expected_root='board', effective_report_path=flat)
result = protocol.read_ascii_discovery(tmp_path / 'board.prj', hierarchy, ('NEW',),
expected_root='board', effective_report_path=flat)
assert [(row.uid, row.refdes, row.schematic_name, row.sheet_number) for row in result.components] == [
('@REFDES|R1' if repeated_uid else '$27I100', 'R1', 'board', 'MAIN')]
child = protocol.read_ascii_discovery(tmp_path / 'board.prj', hierarchy, ('CHILD',),
expected_root='board', effective_report_path=flat)
assert [(row.refdes, row.schematic_name, row.sheet_number) for row in child.components] == [('R2', 'child', 'AUX')]
def test_inventory_uses_effective_instance_part_and_value_before_filtering(tmp_path):
@@ -129,6 +125,11 @@ def test_filtered_or_ambiguous_ascii_report_requires_com_fallback(tmp_path, chan
old, new = substitutions[change]
text = text.replace(old, new)
path.write_text(text)
if change == 'bad_uid':
result = protocol.read_ascii_discovery(tmp_path / 'source.prj', path, ('OLD',),
effective_report_path=effective_report(tmp_path, part='OLD', value='10K/NC'))
assert {row.refdes for row in result.components} == {'R1', 'R2'}
return
with pytest.raises(protocol.NativeInventoryUnsupported):
protocol.read_ascii_discovery(tmp_path / 'source.prj', path, ('OLD',),
effective_report_path=effective_report(tmp_path, part='OLD', value='10K/NC'))
@@ -179,3 +180,41 @@ def test_unrequested_reference_override_does_not_block_certified_target_inventor
with pytest.raises(protocol.NativeInventoryUnsupported, match='位号与属性不一致'):
protocol.read_ascii_discovery(tmp_path / 'board.prj', report(tmp_path), ('OLD',),
effective_report_path=flat)
@pytest.mark.parametrize('uid', ['VCC', '$29I300', '$27I100'])
@pytest.mark.parametrize('part', ['NEW', 'UNREQUESTED'])
def test_explicit_non_pcb_symbols_are_not_replacement_identities(tmp_path, uid, part):
hierarchy, flat = report(tmp_path), effective_report(tmp_path)
# Electrical annotations may carry library properties and reuse names/UIDs.
props = '\tSymbol name power.1 type Primitive\n\tProp Part Number ' + part + '\n'
hierarchy.write_text(hierarchy.read_text() + (' Symbol ' + uid + '\tpage AUX\n' + props) * 2)
flat.write_text(flat.read_text() + (' Symbol ' + uid + '\tref des \tpage \n\tPath \n' + props) * 2)
result = protocol.read_ascii_discovery(tmp_path / 'board.prj', hierarchy, ('NEW',),
effective_report_path=flat)
assert [(row.uid, row.refdes) for row in result.components] == [('$27I100', 'R1')]
@pytest.mark.parametrize('mode', ['hierarchy', 'effective'])
@pytest.mark.parametrize('damage', ['uid', 'ref', 'classification'])
def test_non_pcb_filter_keeps_real_component_identity_checks(tmp_path, mode, damage):
hierarchy, flat = report(tmp_path), effective_report(tmp_path)
flat.write_text(flat.read_text().replace('\tPath \n',
'\tPath \n\tSymbol name res.1 type Primitive Frozen ForwardPCB\n'))
path = hierarchy if mode == 'hierarchy' else flat
text = path.read_text()
if damage == 'uid':
text = text.replace('$27I100', 'MISSING_UID')
elif damage == 'ref':
text = text.replace('\tProp Ref Designator R1\n', '')
else:
text = text.replace('res.1 type Primitive Frozen ForwardPCB', 'power.1 type Primitive', 1)
path.write_text(text)
if damage == 'uid':
result = protocol.read_ascii_discovery(tmp_path / 'board.prj', hierarchy, ('NEW',),
effective_report_path=flat)
assert [(row.uid,row.refdes) for row in result.components] == [('@REFDES|R1','R1')]
return
with pytest.raises(protocol.NativeInventoryUnsupported):
protocol.read_ascii_discovery(tmp_path / 'board.prj', hierarchy, ('NEW',),
effective_report_path=flat)
+3 -3
View File
@@ -23,9 +23,9 @@ def test_normal_replacement_requires_the_selected_library_identity(change):
native.replacement_matches([current], RECORDS, 'NEW', True, library_row=LIBRARY)
def test_recovery_never_substitutes_a_different_uid_with_the_same_refdes():
with pytest.raises(ValueError, match='identity|UID'):
native.existing_replacement_uids([component(uid='$2I999')], RECORDS, 'NEW', {'library_row': LIBRARY})
def test_recovery_reassociates_uid_changes_by_the_same_unique_refdes():
assert native.existing_replacement_uids([component(uid='$2I999')], RECORDS, 'NEW',
{'library_row': LIBRARY}) == ['$2I100']
def test_recovery_treats_part_label_case_as_significant():
+6 -4
View File
@@ -170,7 +170,7 @@ def test_resume_reuses_intact_before_export_but_rebuilds_changed_output(tmp_path
result = {'status': 'success', 'complete': True, 'capture': saved,
'outputs': {'effective': str(output)},
'runs': [{'command': [sys.executable], 'tool_sha256': native_netlist._hash(Path(sys.executable))}],
'graph': {'components': {'u1': {'attributes': {'Part Number': part, 'Value': 'V'}}},
'graph': {'components': {'u1': {'refdes': 'R1', 'attributes': {'Part Number': part, 'Value': 'V'}}},
'nets': [{'name': 'GND', 'endpoints': [['u1', '1']]}]}}
write_record(directory / 'netlist.json', result)
return result
@@ -232,7 +232,7 @@ def test_graph_match_does_not_claim_failed_execution_verified(tmp_path, monkeypa
destination.mkdir(parents=True)
return {'project': str(source), 'root': str(destination)}
graph = {'components': {'u': {'attributes': {'Part Number': 'NEW', 'Value': 'V'}}},
graph = {'components': {'u': {'refdes': 'R1', 'attributes': {'Part Number': 'NEW', 'Value': 'V'}}},
'nets': [{'name': 'N', 'endpoints': [['u', '1']]}]}
monkeypatch.setattr(native_netlist, 'capture_project', capture)
monkeypatch.setattr(native_netlist, 'export_netlist', lambda *a, **k: {'status': 'success', 'complete': True, 'graph': graph})
@@ -253,7 +253,8 @@ def test_expected_library_identity_comes_from_intact_bound_native_receipt(proof_
receipt = {'native_state': 'saved_ok', 'proof': proof}
library = library_from_receipt(command, receipt)
assert library == {'uid': {'attributes': {'Part Number': 'NEW', 'Part Name': 'name', 'Part Label': 'label'},
'target_symbol': 'sym', 'slot_count': 1}}
'target_symbol': 'sym', 'slot_count': 1, 'refdes': 'R1',
'schematic_name': 'MAIN', 'sheet_number': '1'}}
native = next(Path(item['path']) for item in proof['files'] if item['path'].endswith('.native.json'))
native.write_text('{}', encoding='utf-8')
with pytest.raises(ValueError, match='证据'):
@@ -339,7 +340,8 @@ def test_rule_netlist_accepts_bound_component_expansion_and_rejects_foreign_scop
_rule_rows(child, 'task', plan, before)
else:
rows, scopes = _rule_rows(child, 'task', plan, before)
assert rows[0]['target'] == {'uid': 'uid', 'old_value': 'ORIGINAL'}
assert rows[0]['target'] == {'uid': 'uid', 'refdes': 'R1', 'old_value': 'ORIGINAL',
'schematic_name': 'MAIN', 'sheet_number': '1'}
assert rows[0]['new_part_number'] == 'NEW'
assert scopes[0]['source_uids'] == ['uid']
finally:
+12 -9
View File
@@ -13,7 +13,8 @@ from dx_productivity.trial_task_center.target_plan import ComponentTarget, Targe
from dx_productivity.trial_task_center.task_queue import execute_task_queue
def test_hundred_uid_batch_keeps_93_verified_items_and_stops_at_unknown_write(tmp_path):
def test_hundred_uid_batch_keeps_93_verified_items_and_isolates_unknown_writes(tmp_path):
from test_project_isolation_queue import ReleasedRuntime
calls = []
outcomes = {}
def worker(command, timeout):
@@ -58,20 +59,22 @@ def test_hundred_uid_batch_keeps_93_verified_items_and_stops_at_unknown_write(tm
# the production queue, receipts, backups, issue pass and runtime cleanup.
def supervise(operation, arguments, *, stop_event):
assert operation == 'execute_task_queue'
return execute_task_queue(engine, stop_event=stop_event, **arguments)
runtime = ReleasedRuntime(data)
runtime.desktop = None
return execute_task_queue(engine, stop_event=stop_event, runtime=runtime, **arguments)
gui._supervise = supervise
ids = tuple(task.task_id for task in tasks)
try:
result = execute_task_queue(gui, ids, stop_event=Event())
assert result.paused
assert [entry.state for entry in result.entries] == ['failed', 'completed', 'blocked', 'pending']
assert len(calls) == 99 and set(Counter(calls).values()) == {1}
assert (tasks[3].task_id, 'u1') not in calls
assert not result.paused
assert [entry.state for entry in result.entries] == ['failed', 'completed', 'blocked', 'blocked']
assert len(calls) == 100 and set(Counter(calls).values()) == {1}
assert (tasks[3].task_id, 'u1') in calls
records = dict(store._connection.execute('SELECT state,count(*) FROM execution_operations GROUP BY state'))
assert records == {'verified': 93, 'failed': 5, 'uncertain': 1}
assert records == {'verified': 93, 'failed': 5, 'uncertain': 2}
resumed = execute_task_queue(gui, ids, stop_event=Event(), queue_id=result.queue_id)
assert resumed.paused and resumed.entries[2].state == 'blocked'
assert len(calls) == 99
assert not resumed.paused and resumed.entries[2].state == 'blocked'
assert len(calls) == 100
finally:
store.close()
reopened = TaskStore(database)
+7 -3
View File
@@ -88,9 +88,13 @@ def test_uncertain_component_never_runs_same_project_remainder(component_case):
service, _, tasks, calls, outcomes = component_case
task_id = tasks[0].task_id
outcomes[(task_id, ('R1',))] = 'batch_saved_unverified'
result = execute_task_queue(service, (task_id,), stop_event=Event())
assert result.paused
assert calls == [(task_id, ('R1',))]
ids = tuple(task.task_id for task in tasks)
result = execute_task_queue(service, ids, stop_event=Event())
assert not result.paused and [entry.state for entry in result.entries] == ['blocked', 'completed']
assert calls == [(task_id, ('R1',)), (ids[1], ('R1',)), (ids[1], ('R2',))]
assert service.has_interrupted_quick_run(task_id)
execute_task_queue(service, ids, stop_event=Event(), queue_id=result.queue_id)
assert len(calls) == 3
def test_problem_retry_budget_survives_rebinding_and_is_shared(component_case):
+88
View File
@@ -2,6 +2,94 @@ import pytest
import test_accuracy_vbs_guards as guards
@pytest.mark.parametrize('fault', ['', 'snapshot', 'identity', 'transaction', 'native',
'save_requested', 'save_committed', 'close'])
def test_safe_pause_cannot_interrupt_verified_failure_cleanup(tmp_path, fault):
output = guards.run_vbs(tmp_path, ['QPProgress', 'QPDiscardFailedBatchProject',
'QPCloseBatchProject', 'QPReadUtf8'], '''
Dim Failure,Stream,PausedBefore,PausedAfter
Set QP_App=New FakeApp
Set Stream=QP_Fso.CreateTextFile(QP_Fso.BuildPath(QP_Root,"monitor-token.stop"),True)
Stream.Write "token":Stream.Close
On Error Resume Next
QPProgress "query_components",0,1
PausedBefore=(Err.Source="QuickPartReplace.SafeStop"):Err.Clear
QPDiscardFailedBatchProject Failure
If Err.Number<>0 Then WScript.Quit 8
QPProgress "query_components",0,1
PausedAfter=(Err.Source="QuickPartReplace.SafeStop"):Err.Clear
WScript.Echo "result:" & CStr(PausedBefore) & ":" & CStr(QP_App.Closed) & ":" & _
CStr(Len(Failure)>0) & ":" & CStr(PausedAfter)
''', prelude='''
Option Explicit
Dim QP_Fso,QP_Root,QP_App,QP_CurrentJobId,QP_AttemptToken,QP_ProgressSequence
Dim QP_NativeMutationPossible,QP_RestoreTransactionOpen,QP_NativeCallActive,QP_ExpectedJobId
Dim QP_BatchMode,QP_SessionMode,QP_MonitorJobId,QP_SessionToken,QP_OwnedProject,QP_OwnedProjectPath
Dim QP_BatchSaveRequested,QP_BatchSaveCommitted,QP_SessionExternalChange,QP_FailureCleanupActive
Dim QP_SessionItems,QP_SessionDocuments
Set QP_SessionItems=CreateObject("Scripting.Dictionary")
Set QP_SessionDocuments=CreateObject("Scripting.Dictionary")
Set QP_Fso=CreateObject("Scripting.FileSystemObject")
QP_Root="''' + str(tmp_path) + '''":QP_AttemptToken="token":QP_ProgressSequence=0
QP_BatchMode=True:QP_OwnedProject=True:QP_OwnedProjectPath="board.prj"
QP_NativeMutationPossible=''' + str(fault == 'native') + '''
QP_BatchSaveRequested=''' + str(fault == 'save_requested') + '''
QP_BatchSaveCommitted=''' + str(fault == 'save_committed') + '''
Class FakeApp
Public Closed,Path
Private Documents
Private Sub Class_Initialize()
Closed=False:Path="board.prj"
Set Documents=CreateObject("Scripting.Dictionary")
Documents.Add 1,New FakeDocument
End Sub
Function SchematicSheetDocuments()
Set SchematicSheetDocuments=Documents
End Function
Sub RemoveDocument()
Documents.Remove 1
End Sub
Sub CloseProject()
Closed=True:Path=""
End Sub
Function GetProjectData()
Set GetProjectData=Me
End Function
Function GetProjectFilePath()
GetProjectFilePath=Path
End Function
End Class
Class FakeDocument
Sub Close(ByVal Save,ByVal Name)
If Save Then Err.Raise 1235,"Test","unexpected save"
If "''' + fault + '''"="close" Then Err.Raise 1236,"FakeCOM","close failed"
QP_App.RemoveDocument
End Sub
End Class
Sub QPVerifyFailedOriginalSnapshot()
If "''' + fault + '''"="snapshot" Then Err.Raise 1234,"Snapshot","source changed"
If "''' + fault + '''"="identity" Then QP_App.Path="other.prj"
If "''' + fault + '''"="transaction" Then QP_RestoreTransactionOpen=True
QPProgress "verify_untouched",1,1
End Sub
Function QPCanonicalPath(ByVal Path)
QPCanonicalPath=LCase(Path)
End Function
Function QPProjectPath()
QPProjectPath=QP_App.Path
End Function
Sub QPInvalidatePinIndex()
End Sub
Sub QPInvalidateComponentIndex()
End Sub
Sub QPLog(ByVal Root,ByVal Level,ByVal Message)
End Sub
''')
paused_before = fault != 'native'
paused_after = fault not in {'native', 'transaction'}
assert output.splitlines()[-1] == f'result:{paused_before}:{not fault}:{bool(fault)}:{paused_after}'
@pytest.mark.parametrize('snapshot_changed', [False, True])
def test_failed_project_closes_only_after_unchanged_snapshot_is_certified(tmp_path, snapshot_changed):
output = guards.run_vbs(tmp_path, ['QPDiscardFailedBatchProject'], '''
@@ -0,0 +1,171 @@
from contextlib import contextmanager
from pathlib import Path
import subprocess
from threading import Event
from types import SimpleNamespace
import pytest
from dx_productivity.trial_task_center.logging_setup import configure_logging
from dx_productivity.trial_task_center.quick_run import QuickRunPreparer, _encode_vbs_field
from dx_productivity.trial_task_center.service import TaskCenterService
from dx_productivity.trial_task_center.store import TaskStore
@pytest.mark.parametrize('pause_after_first', [False, True])
def test_real_batch_preparer_and_journal_commit_staged_prefix_only_after_native_close(tmp_path, pause_after_first, monkeypatch):
monkeypatch.setattr('dx_productivity.trial_task_center.scanner._is_schematic_project', lambda path: True)
project = tmp_path / 'projects' / 'board.prj'
project.parent.mkdir(); project.write_text('disposable project')
root = tmp_path / 'data' / 'quick-run'
stop = Event()
preparer = QuickRunPreparer(tmp_path / 'data', process_runner=lambda *args: None,
cscript_resolver=lambda: Path('cscript.exe'), sheet_discoverer=lambda project: (('Main', '1'),))
service = TaskCenterService(TaskStore(tmp_path / 'tasks.db'), configure_logging(tmp_path / 'data'), quick_run_preparer=preparer)
task = service.create_task('staged project')
service.scan_projects(task.task_id, project.parent)
for row in service.projects(task.task_id):service.set_project_selected(row.project_id, True)
service.add_rule(task.task_id, 'OLD', 'NEW')
service.add_rule(task.task_id, 'OTHER', 'NEW')
class NativeBoundary:
enabled = True
worker = None
on_commit = None
jobs = []
closes = 0
def prepare(self, project):pass
def run_step(self, project, command, timeout):
self.worker = SimpleNamespace(token='a' * 32, root=root)
job = command[-1]
if self.jobs:
states = [row[0] for row in service._store._connection.execute('SELECT state FROM execution_operations')]
assert states.count('pending_save') == 1
assert 'verified' not in states
assert self.closes == 0
self.jobs.append(job)
with (root / 'status.tsv').open('a', encoding='utf-8') as stream:
stream.write(f'{job}\tpending_save\tsession_staged_no_match\tverified in memory\n')
if pause_after_first:stop.set()
return subprocess.CompletedProcess(command, 0, '', '')
def close(self):
if self.worker is None:return
self.closes += 1
marker = f'{self.worker.token}\t{project}\tproject_verified\n'
for job in self.jobs:
state = '\t'.join(_encode_vbs_field(str(value)) for value in
(job, project, 'Main', '1', 'S\tfull_sheet_v1\r\nunchanged'))
(root / f'post-state-{job}.tsv').write_text(
'job_id\tproject_path\tschematic_name\tsheet_number\tsemantic_state\n' + state + '\n', encoding='utf-8')
marker += f'{job}\tno_match\n'
(root / f'project-commit-{self.worker.token}.tsv').write_text(marker, encoding='utf-8')
with (root / 'status.tsv').open('a', encoding='utf-8') as stream:
for job in self.jobs:stream.write(f'{job}\tno_match\tproject_committed\tclosed and read back\n')
if self.on_commit:self.on_commit(self.worker)
self.worker = None
boundary = NativeBoundary()
@contextmanager
def retained_scope(**kwargs):
try:yield boundary
finally:boundary.close()
preparer.project_session_scope = retained_scope
progress = service.quick_run_progress(task.task_id)
result = service.execute_quick_run_queue(task.task_id,
expected_item=(str(project), progress.next_old_part_number, progress.next_new_part_number, 'preserve'), stop_event=stop)
assert boundary.closes == 1
assert len(boundary.jobs) == (1 if pause_after_first else 2)
assert result.progress.completed_items == len(boundary.jobs)
assert result.progress.saved_items == 0
assert result.progress.no_match_items == len(boundary.jobs)
assert result.paused is pause_after_first
assert result.progress.queue_complete is (not pause_after_first)
assert service._store._connection.execute('SELECT state FROM execution_project_commits').fetchone()[0] == 'verified'
states = [row[0] for row in service._store._connection.execute('SELECT state FROM execution_operations')]
assert states.count('no_match') == len(boundary.jobs)
if pause_after_first:assert states.count('pending') == 1
service._store.close()
def test_runtime_prewrite_pause_commits_prefix_and_retires_only_current_claim(tmp_path, monkeypatch):
from test_batch_runtime import worker_runtime, Cad
from dx_productivity.trial_task_center.errors import TaskCenterError
from dx_productivity.trial_task_center.stop_control import StopControl
monkeypatch.setattr('dx_productivity.trial_task_center.scanner._is_schematic_project', lambda path: True)
project = tmp_path / 'projects' / 'board.prj'
project.parent.mkdir(); project.write_text('disposable project')
root = tmp_path / 'data' / 'quick-run'
stop = StopControl()
preparer = QuickRunPreparer(tmp_path / 'data', process_runner=lambda *args: None,
cscript_resolver=lambda: Path('cscript.exe'), sheet_discoverer=lambda project: (('Main', '1'),))
service = TaskCenterService(TaskStore(tmp_path / 'tasks.db'), configure_logging(tmp_path / 'data'), quick_run_preparer=preparer)
task = service.create_task('prewrite pause')
service.scan_projects(task.task_id, project.parent)
for row in service.projects(task.task_id):service.set_project_selected(row.project_id, True)
service.add_rule(task.task_id, 'OLD', 'NEW'); service.add_rule(task.task_id, 'OTHER', 'NEW')
token = 'a' * 32
current = {}
def native_monitor(command, *args, **kwargs):
stop.set()
prefix, paused = current['jobs']
fields = (prefix, str(project), 'Main', '1', 'S\tfull_sheet_v1\r\nverified final state')
(root / f'post-state-{prefix}.tsv').write_text(
'job_id\tproject_path\tschematic_name\tsheet_number\tsemantic_state\n' +
'\t'.join(_encode_vbs_field(value) for value in fields) + '\n', encoding='utf-8')
(root / f'project-commit-{token}.tsv').write_text(f'{token}\t{project}\tproject_verified\n{prefix}\tno_match\n', encoding='utf-8')
(root / f'project-pause-{token}.tsv').write_text(f'{token}\t{project}\t{paused}\tprewrite_closed', encoding='utf-8')
with (root / 'status.tsv').open('a', encoding='utf-8') as stream:
stream.write(f'{prefix}\tno_match\tproject_committed\tprefix closed\n')
stream.write(f'{paused}\tawaiting_native_replace\t\tsnapshot before pause\n')
result = subprocess.CompletedProcess(command, 75, '', '')
result.prewrite_pause = dict(session_token=token, monitor_token='b' * 32, job_id=paused,
project_path=str(project), sequence=7, worker_tree_exited=True)
return result
runtime = worker_runtime(tmp_path, monkeypatch, native_monitor)
runtime.cad_factory = lambda *args, stop_control=None, **options: Cad(*args, **options)
runtime.stop_event = stop
class RetainedNative:
enabled = True
worker = SimpleNamespace(token=token, root=root)
on_commit = None
def prepare(self, project):pass
def run_step(self, project, command, timeout):
current.setdefault('jobs', []).append(command[-1])
(root / 'project-session.request').write_text(f'{token}\t{project}\t{command[-1]}', encoding='utf-8')
if len(current['jobs']) == 1:
current['initial_command'] = command
with (root / 'status.tsv').open('a', encoding='utf-8') as stream:
stream.write(f'{command[-1]}\tpending_save\tsession_staged_no_match\tstaged\n')
return subprocess.CompletedProcess(command, 0, '', '')
return runtime.run_worker(current['initial_command'], timeout)
def close(self):
if self.on_commit:self.on_commit(self.worker)
self.worker = None
boundary = RetainedNative()
@contextmanager
def retained_scope(**kwargs):
try:yield boundary
finally:boundary.close()
preparer.project_session_scope = retained_scope
progress = service.quick_run_progress(task.task_id)
with runtime, pytest.raises(TaskCenterError) as raised:
service.execute_quick_run_queue(task.task_id, expected_item=(str(project), progress.next_old_part_number,
progress.next_new_part_number, 'preserve'), stop_event=stop, runtime=runtime)
assert raised.value.code == 'operation_cancelled'
assert not (root / 'uncertain.flag').exists()
assert not (root / 'current-command.tsv').exists()
states = [row[0] for row in service._store._connection.execute('SELECT state FROM execution_operations ORDER BY rowid')]
assert states == ['no_match', 'pending']
record = service._store._connection.execute("SELECT * FROM execution_operations WHERE state='pending'").fetchone()
assert record['generation'] == 0
assert preparer.journal.certified_startup_pause(task.task_id, record)
assert service.quick_run_progress(task.task_id).completed_items == 1
assert not preparer.has_interrupted(task.task_id)
service._store.close()
+108
View File
@@ -0,0 +1,108 @@
import hashlib
import json
import pytest
from dx_productivity.trial_task_center.errors import TaskCenterError
from dx_productivity.trial_task_center.execution_journal import ExecutionJournal, create_execution_schema
from dx_productivity.trial_task_center.history_store import HistoryRepository, create_history_schema
from dx_productivity.trial_task_center.store import TaskStore
@pytest.fixture
def staged_case(tmp_path):
store = TaskStore(tmp_path / 'tasks.db')
with store.transaction():
create_execution_schema(store._connection)
create_history_schema(store._connection)
task = store.create_task('project commit')
journal = ExecutionJournal(store)
rows = [dict(task_id=task.task_id, job_id=f'job{index}', project_path=str(tmp_path / 'board.prj'),
schematic_name='MAIN', sheet_number='1', old_part='OLD', new_part='NEW',
target_scope='targets', target_selectors=f'uid{index}\x1fR{index}\x1f10K') for index in (1, 2)]
journal.bind(task.task_id, rows, {})
evidence = tmp_path / 'readback'; evidence.write_text('persisted sheet state')
proof = {'files': [{'path': str(evidence), 'sha256': hashlib.sha256(evidence.read_bytes()).hexdigest()}],
'baseline': dict(project_path=rows[0]['project_path'], schematic_name='MAIN', sheet_number='1', state='persisted sheet state')}
yield store, journal, task.task_id, rows, proof
store.close()
def test_staged_prefix_is_not_saved_and_cannot_replay_after_rebind(staged_case, tmp_path):
store, journal, task, rows, proof = staged_case
attempt = journal.claim(task, rows[0])
journal.stage_project_result(attempt, rows[0], session_token='a' * 32, root=tmp_path, proof=proof)
record = store._connection.execute('SELECT state FROM execution_operations WHERE attempt_id=?', (attempt.attempt_id,)).fetchone()
assert record['state'] == 'pending_save'
assert HistoryRepository(store).page().rows == ()
with pytest.raises(TaskCenterError, match='quick_run_uncertain'):
journal.bind(task, rows, {rows[0]['job_id']: ('saved_ok', 'session_verified', 'uncommitted')})
assert journal.needs_reconciliation(task)
assert journal.unresolved_jobs(task, rows) == ('job1',)
# The active retained owner can stage another independent operation.
assert journal.claim(task, rows[1]).generation == 1
def test_project_commit_is_atomic_idempotent_and_preserves_operation_ids(staged_case, tmp_path, monkeypatch):
store, journal, task, rows, proof = staged_case
attempts = [journal.claim(task, row) for row in rows]
for attempt, row in zip(attempts, rows):
journal.stage_project_result(attempt, row, session_token='a' * 32, root=tmp_path, proof=proof)
commit_proof = dict(proof, session_token='a' * 32, project_path=rows[0]['project_path'], jobs=['job1', 'job2'])
original = HistoryRepository.append
calls = []
def fail_second(self, **values):
calls.append(values)
if len(calls) == 2:
raise OSError('disk full')
return original(self, **values)
monkeypatch.setattr(HistoryRepository, 'append', fail_second)
with pytest.raises(OSError):
journal.commit_project_results(task, 'a' * 32, {'job1': proof, 'job2': proof}, commit_proof)
assert [r[0] for r in store._connection.execute('SELECT state FROM execution_operations')] == ['pending_save', 'pending_save']
assert HistoryRepository(store).page().rows == ()
monkeypatch.setattr(HistoryRepository, 'append', original)
journal.commit_project_results(task, 'a' * 32, {'job1': proof, 'job2': proof}, commit_proof)
journal.commit_project_results(task, 'a' * 32, {'job1': proof, 'job2': proof}, commit_proof)
assert len(HistoryRepository(store).page().rows) == 2
records = store._connection.execute('SELECT state,attempt_id,generation FROM execution_operations').fetchall()
assert [r['state'] for r in records] == ['verified', 'verified']
assert [r['attempt_id'] for r in records] == [a.attempt_id for a in attempts]
assert journal.bind(task, rows, {})['job1'][0] == 'saved_ok'
def test_partial_pause_commits_only_staged_prefix_and_rejects_incomplete_readback(staged_case, tmp_path):
store, journal, task, rows, proof = staged_case
attempt = journal.claim(task, rows[0])
journal.stage_project_result(attempt, rows[0], session_token='a' * 32, root=tmp_path, proof=proof)
with pytest.raises(TaskCenterError):
journal.commit_project_results(task, 'a' * 32, {}, dict(proof, jobs=[]))
commit = dict(proof, session_token='a' * 32, project_path=rows[0]['project_path'], jobs=['job1'])
journal.commit_project_results(task, 'a' * 32, {'job1': proof}, commit)
assert journal.claim(task, rows[1]).generation == 1
record = store._connection.execute('SELECT state FROM execution_project_commits').fetchone()
assert record['state'] == 'verified'
def test_crash_reconciliation_commits_native_marker_without_replaying_any_worker(staged_case, tmp_path):
from dx_productivity.trial_task_center.quick_run import QuickRunPreparer, _encode_vbs_field
store, journal, task, rows, proof = staged_case
attempt = journal.claim(task, rows[0])
journal.stage_project_result(attempt, rows[0], session_token='a' * 32, root=tmp_path, proof=proof)
def forbidden_worker(*args):
pytest.fail('recovery replayed native replacement')
preparer = QuickRunPreparer(tmp_path, process_runner=forbidden_worker)
preparer.journal = journal
with pytest.raises(TaskCenterError):preparer._commit_staged_project(task)
assert store._connection.execute('SELECT state FROM execution_operations WHERE attempt_id=?', (attempt.attempt_id,)).fetchone()[0] == 'pending_save'
job = rows[0]['job_id']
fields = (job, rows[0]['project_path'], 'MAIN', '1', 'S\tfull_sheet_v1\r\nfinal persisted state')
(tmp_path / f'post-state-{job}.tsv').write_text(
'job_id\tproject_path\tschematic_name\tsheet_number\tsemantic_state\n' +
'\t'.join(_encode_vbs_field(value) for value in fields) + '\n', encoding='utf-8')
(tmp_path / ('project-commit-' + 'a' * 32 + '.tsv')).write_text(
'a' * 32 + '\t' + rows[0]['project_path'] + '\tproject_verified\n' + job + '\tno_match\n', encoding='utf-8')
preparer._commit_staged_project(task)
preparer._commit_staged_project(task)
assert store._connection.execute('SELECT state FROM execution_operations WHERE attempt_id=?', (attempt.attempt_id,)).fetchone()[0] == 'no_match'
assert len(HistoryRepository(store).page().rows) == 1
@@ -0,0 +1,27 @@
from pathlib import Path
from types import SimpleNamespace
from dx_productivity.trial_task_center.quick_run import QuickRunProgress, _advance_batch_session, _correlated_status_change
def test_staging_advances_live_cursor_without_counting_saved_or_completed_items(tmp_path):
project = str(tmp_path / 'board.prj')
commands = (('task', 'one', project, 'OLD', 'NEW'), ('task', 'two', project, 'OTHER', 'NEW'))
progress = QuickRunProgress(0, 2, Path(project), 'OLD', 'NEW', (), False)
session = SimpleNamespace(commands=commands, next_index=0, latest={}, progress=progress, sheets=None)
_advance_batch_session(session, ('one', 'pending_save', 'session_staged', 'staged'))
assert session.next_index == 1
assert session.progress.completed_items == 0
assert session.progress.saved_items == 0
assert session.progress.completed_project_paths == ()
assert session.progress.next_old_part_number == 'OTHER'
assert not session.progress.queue_complete
def test_native_staged_and_resumed_pause_statuses_correlate_to_current_operation():
command = ('task', 'one', 'board.prj', 'OLD', 'NEW')
waiting = ('one', 'awaiting_native_replace', '', 'snapshot')
staged = ('one', 'pending_save', 'session_staged', 'verified in memory')
assert _correlated_status_change((), (waiting, staged), (command,), expected_command=command) == staged
paused = ('one', 'pending', 'prewrite_paused', 'not replaced')
assert _correlated_status_change((paused,), (paused, waiting, staged), (command,), expected_command=command) == staged
+29
View File
@@ -0,0 +1,29 @@
import json
import pytest
from dx_productivity.trial_task_center.errors import TaskCenterError
from dx_productivity.trial_task_center.execution_proof import capture_project_commit, proof_intact
def test_commit_manifest_binds_ordered_prefix_project_and_immutable_copy(tmp_path):
root = tmp_path / 'worker'; root.mkdir()
project = tmp_path / 'board.prj'
token = 'a' * 32
commit = dict(session_token=token, project_path=str(project.resolve()).casefold(), jobs_json=json.dumps(['one', 'two']))
marker = root / f'project-commit-{token}.tsv'
marker.write_text(f'{token}\t{project}\tproject_verified\none\tsaved_ok\ntwo\tno_match\n', encoding='utf-8')
proof = capture_project_commit(tmp_path, root, commit)
assert proof['jobs'] == ['one', 'two']
assert proof['outcomes'] == {'one': 'saved_ok', 'two': 'no_match'}
marker.write_text('crashed next session', encoding='utf-8')
assert proof_intact(proof)
@pytest.mark.parametrize('tail', ['one\tsaved_ok\n', 'two\tsaved_ok\none\tsaved_ok\n', 'one\tpending_save\ntwo\tsaved_ok\n'])
def test_commit_manifest_rejects_partial_or_unverified_prefix(tmp_path, tail):
project = tmp_path / 'board.prj'; token = 'a' * 32
commit = dict(session_token=token, project_path=str(project.resolve()).casefold(), jobs_json=json.dumps(['one', 'two']))
(tmp_path / f'project-commit-{token}.tsv').write_text(f'{token}\t{project}\tproject_verified\n' + tail, encoding='utf-8')
with pytest.raises(TaskCenterError):
capture_project_commit(tmp_path, tmp_path, commit)
@@ -0,0 +1,216 @@
import json
from pathlib import Path
import pytest
from dx_productivity.trial_task_center.errors import TaskCenterError
from dx_productivity.trial_task_center.history_store import HistoryRepository
from dx_productivity.trial_task_center.logging_setup import configure_logging
from dx_productivity.trial_task_center.models import TaskStatus
from dx_productivity.trial_task_center.native_helper import read_snapshot, transaction_binding
from dx_productivity.trial_task_center.quick_run import QuickRunPreparer, _encode_vbs_field as enc, _write_uncertain_flag, _run_process
from dx_productivity.trial_task_center.service import TaskCenterService
from dx_productivity.trial_task_center.store import TaskStore
@pytest.fixture
def public_commit_case(tmp_path, monkeypatch, request):
monkeypatch.setattr('dx_productivity.trial_task_center.scanner._is_schematic_project', lambda _: True)
project = tmp_path / 'project' / 'board.prj'
project.parent.mkdir(); project.write_text('saved both replacements')
data = tmp_path / 'data'
preparer = QuickRunPreparer(data, process_runner=lambda *args: pytest.fail('native replacement/recovery replayed'),
cscript_resolver=lambda: Path('cscript.exe'), sheet_discoverer=lambda _: (('MAIN', '1'),))
store = TaskStore(tmp_path / 'tasks.db')
service = TaskCenterService(store, configure_logging(data), quick_run_preparer=preparer)
task = service.create_task('saved engineering project')
service.scan_projects(task.task_id, project.parent)
for row in service.projects(task.task_id):service.set_project_selected(row.project_id, True)
service.add_rule(task.task_id, 'OLD1', 'NEW1'); service.add_rule(task.task_id, 'OLD2', 'NEW2')
root = data / 'quick-run'
handoff = preparer.prepare(task.task_id, service.projects(task.task_id), service.rules(task.task_id),
sheet_targets={str(project.resolve()).casefold(): (('MAIN', '1'),)})
rows = list(preparer._bind_journal_handoff(task.task_id, handoff).values())
preparer._process_runner = _run_process
attempts = []
for index, row in enumerate(rows, 1):
job = row['job_id']; uid = f'original{index}'; ref = f'R{index}'
snapshot = root / f'snapshot-{job}.tsv'
snapshot.write_text('\t'.join(map(enc, ['M', str(project), row['old_part'], row['new_part'], 'MAIN:1', 'preserve', '', '1'])) + '\n' +
f'C\t{uid}\t{ref}\t{index*100}\t0\t0\t1\tNO_PINS\nA\t{uid}\tValue\t1K\t1K\t0\t0\t0\t0\t1\t0\t1\n', encoding='utf-8')
metadata, components = read_snapshot(snapshot)
report = {'ok': True, 'stage': 'native_replace_verified',
'transaction': transaction_binding(snapshot, metadata, components, ['lib', row['new_part'], 'symbol', '', 'name', 'label'], 'symbol'),
'batch_attempt': dict(task_id=task.task_id, job_id=job, project_path=str(project), session_id='b'*32, attempt_id='c'*32, generation=1)}
Path(str(snapshot) + '.native.json').write_text(json.dumps(report), encoding='utf-8')
(root / f'post-state-{job}.tsv').write_text('job_id\tproject_path\tschematic_name\tsheet_number\tsemantic_state\n' +
'\t'.join(map(enc, [job, str(project), 'MAIN', '1', 'S\tfull_sheet_v1\r\nR1=NEW1\r\nR2=NEW2\r\n'])) + '\n', encoding='utf-8')
raw = [['V','2',task.task_id,job,'saved_ok',str(project),'MAIN','1',row['old_part'],row['new_part'],
'preserve','','1','','',snapshot.read_bytes().decode('utf-8')], ['C',uid,ref,row['new_part'],'1K','1','']]
(root / f'value-readback-{job}.tsv').write_text('\n'.join('\t'.join(map(enc, fields)) for fields in raw) + '\n', encoding='utf-8')
attempt = preparer.journal.claim(task.task_id, row); attempts.append(attempt)
proof = preparer.result_proof(root, row, ('pending_save', 'session_staged', 'staged'))
preparer.journal.stage_project_result(attempt, row, session_token='a'*32, root=root, proof=proof)
(root / ('project-commit-' + 'a'*32 + '.tsv')).write_text('a'*32 + '\t' + str(project) + '\tproject_verified\n' +
''.join(row['job_id'] + '\tsaved_ok\n' for row in rows), encoding='utf-8')
# Production native status append failed/truncated after the durable marker.
with (root / 'status.tsv').open('a', encoding='utf-8') as stream:stream.write(rows[0]['job_id'] + '\tpending_save\tsession_staged\tstaged\n')
if getattr(request, 'param', None) != 'checkpoint':
original = HistoryRepository.append
def disk_full(*args, **kwargs):raise OSError('history disk full')
monkeypatch.setattr(HistoryRepository, 'append', disk_full)
with pytest.raises(OSError, match='history disk full'):
preparer._commit_staged_project(task.task_id, session_token='a'*32, bind_source=True)
monkeypatch.setattr(HistoryRepository, 'append', original)
_write_uncertain_flag(root / 'uncertain.flag', 'receipt_commit_failed')
store.set_task_status(task.task_id, TaskStatus.BLOCKED, summary='saved project receipt commit failed')
monkeypatch.setattr(TaskCenterService, '_owned_cad_operation', lambda *args, **kwargs: (_ for _ in ()).throw(
TaskCenterError('native_recovery_required', 'ordinary recovery path requires a separately verified native boundary')))
yield service, store, task.task_id, preparer, project, rows, attempts
store.close()
def test_deleted_task_cannot_recover_bound_project_commit(public_commit_case):
service, store, task, preparer, _, _, _ = public_commit_case
flag = preparer.data_root / 'quick-run' / 'uncertain.flag'
protection = flag.read_bytes()
store.set_task_status(task, TaskStatus.DELETED, summary='deleted by operator')
with pytest.raises(TaskCenterError) as caught:
service.recover_quick_run(task, confirmed=True)
assert caught.value.code == 'quick_run_recovery'
assert caught.value.message == '请先恢复已删除的任务记录,再核验其中断步骤。'
assert flag.read_bytes() == protection
assert store.get_task(task).status is TaskStatus.DELETED
assert [row[0] for row in store._connection.execute('SELECT state FROM execution_operations')] == ['pending_save', 'pending_save']
def test_public_confirmed_project_commit_recovery_is_atomic_without_native_replay(public_commit_case):
service, store, task, preparer, project, rows, attempts = public_commit_case
root = preparer.data_root / 'quick-run'
assert [r[0] for r in store._connection.execute('SELECT state FROM execution_operations')] == ['pending_save', 'pending_save']
with pytest.raises(TaskCenterError):service.recover_quick_run(task, confirmed=False)
assert store.get_task(task).status is TaskStatus.BLOCKED
result = service.recover_quick_run(task, confirmed=True)
assert result.state == 'saved_ok' and result.progress.saved_items == 2
assert store.get_task(task).status is TaskStatus.COMPLETED
assert not (root / 'uncertain.flag').exists()
actual = store._connection.execute('SELECT state,attempt_id,generation FROM execution_operations ORDER BY rowid').fetchall()
assert [r['state'] for r in actual] == ['verified', 'verified']
assert [(r['attempt_id'], r['generation']) for r in actual] == [(a.attempt_id, a.generation) for a in attempts]
assert len([r for r in HistoryRepository(store).page().rows if r['operation'] == '器件替换']) == 2
assert service.recover_quick_run(task, confirmed=True).state == 'saved_ok'
@pytest.mark.parametrize('damage', ['manifest', 'source', 'later_attempt', 'later_job', 'unrelated_reason', 'native_report'])
def test_public_recovery_refuses_unproven_or_later_write_uncertainty(public_commit_case, damage):
service, store, task, preparer, project, rows, _ = public_commit_case
root = preparer.data_root / 'quick-run'
if damage == 'manifest':(root / ('project-commit-' + 'a'*32 + '.tsv')).write_text('partial')
elif damage == 'source':project.write_text('external or later native modification')
elif damage == 'later_attempt':
store._connection.execute("UPDATE execution_operations SET generation=generation+1 WHERE operation_key=(SELECT operation_key FROM execution_operations LIMIT 1)")
elif damage == 'later_job':
record = dict(store._connection.execute('SELECT * FROM execution_operations LIMIT 1').fetchone())
command = json.loads(record['command_json']); command['old_part'] = 'LATER'
store._connection.execute('''INSERT INTO execution_operations(task_id,round_id,operation_key,command_json,state,attempt_id,generation)
VALUES (?,?,?,?, 'prepared',?,1)''', (task, record['round_id'], 'later-operation', json.dumps(command), 'd'*32))
elif damage == 'native_report':(root / f'snapshot-{rows[0]["job_id"]}.tsv.native.json').write_text('{}')
else:_write_uncertain_flag(root / 'uncertain.flag', 'native_failure_after_snapshot')
with pytest.raises((TaskCenterError, AssertionError)):service.recover_quick_run(task, confirmed=True)
assert (root / 'uncertain.flag').exists()
assert store.get_task(task).status is TaskStatus.BLOCKED
def test_public_recovery_after_real_store_and_preparer_restart(public_commit_case):
service, store, task, preparer, project, rows, attempts = public_commit_case
path, data = store.path, preparer.data_root
store.close()
reopened = TaskStore(path)
try:
worker = QuickRunPreparer(data, process_runner=_run_process)
resumed = TaskCenterService(reopened, configure_logging(data), quick_run_preparer=worker)
result = resumed.recover_quick_run(task, confirmed=True)
assert result.progress.saved_items == 2 and reopened.get_task(task).status is TaskStatus.COMPLETED
finally:reopened.close()
@pytest.mark.parametrize('public_commit_case', ['checkpoint'], indirect=True)
@pytest.mark.parametrize('damage', [None, 'missing_checkpoint', 'missing_owner', 'native_attempt', 'readback', 'supervisor', 'foreign_supervisor'])
def test_public_recovery_uses_runtime_checkpoint_when_sql_callback_never_binds(public_commit_case, damage):
from types import SimpleNamespace
from dx_productivity.trial_task_center.batch_runtime import BatchCadRuntime
from test_batch_runtime import Desktop
service, store, task, preparer, project, rows, attempts = public_commit_case
root = preparer.data_root / 'quick-run'
assert preparer.journal.project_commit_records(task)[0]['proof_json'] is None
Desktop.fail_release = False
runtime = BatchCadRuntime(preparer.data_root, 'execution', desktop_factory=Desktop, session_id='b'*32)
runtime._ensure_desktop()
runtime.cad = SimpleNamespace(checkpoint=lambda _: None)
runtime.record['active_job'] = rows[0]['job_id']
runtime._current_attempts = [(root / (row['job_id'] + '.json'), row) for row in rows]
(root / 'project-session.request').write_text('a'*32 + '\t' + str(project) + '\t' + rows[0]['job_id'])
runtime._checkpoint(root, project, 'c'*32)
runtime.desktop.close()
runtime.record['cleanup'] = runtime.desktop.cleanup_record
runtime.desktop = runtime.cad = None
runtime._record_release()
checkpoint_path = root / 'project-checkpoint.json'
if damage == 'missing_checkpoint':checkpoint_path.unlink()
elif damage == 'missing_owner':runtime.path.unlink()
elif damage == 'native_attempt':
checkpoint = json.loads(checkpoint_path.read_text())
checkpoint['project_commit_source']['native_attempt']['attempt_id'] = 'e'*32
checkpoint_path.write_text(json.dumps(checkpoint))
elif damage == 'readback':(root / ('value-readback-' + rows[0]['job_id'] + '.tsv')).write_text('changed')
elif damage in {'supervisor', 'foreign_supervisor'}:
from dx_productivity.trial_task_center.batch_resources import write_record
from dx_productivity.trial_task_center.batch_supervisor import _mark_interrupted
(root / 'uncertain.flag').unlink()
_mark_interrupted(preparer.data_root, {'batch_id': runtime.session_id, 'operation': 'execute_task_queue',
'task_ids': [task], 'uncertainty_roots': {task: str(preparer.data_root)}},
'parent crashed after native checkpoint before SQL callback')
if damage == 'foreign_supervisor':
supervisor_path = runtime.path.parent / 'supervisor.json'
supervisor = json.loads(supervisor_path.read_text()); supervisor['protected_task_ids'] = ['foreign-task']
write_record(supervisor_path, supervisor)
# The SQL callback was never entered; a new process consumes the native certificate.
reopened = TaskStore(store.path)
try:
worker = QuickRunPreparer(preparer.data_root, process_runner=_run_process)
resumed = TaskCenterService(reopened, configure_logging(preparer.data_root), quick_run_preparer=worker)
if damage in {None, 'supervisor'}:
assert resumed.recover_quick_run(task, confirmed=True).progress.saved_items == 2
assert reopened.get_task(task).status is TaskStatus.COMPLETED
else:
with pytest.raises(TaskCenterError):resumed.recover_quick_run(task, confirmed=True)
assert (root / 'uncertain.flag').is_file()
assert reopened.get_task(task).status is TaskStatus.BLOCKED
finally:reopened.close()
@pytest.mark.parametrize('failure', ['status', 'flag', 'sql_status'])
def test_public_recovery_retries_publication_after_sql_commit_without_duplicate_history(public_commit_case, monkeypatch, failure):
from dx_productivity.trial_task_center import quick_run
service, store, task, preparer, project, rows, attempts = public_commit_case
root = preparer.data_root / 'quick-run'
original_status, original_replace = quick_run._write_status_details, Path.replace
count = []
def status(*args, **kwargs):
if failure == 'status' and not count:count.append(1); raise OSError('status publication failed')
return original_status(*args, **kwargs)
def replace(path, target):
if failure == 'flag' and path.name == 'uncertain.flag' and not count:count.append(1); raise OSError('flag retirement failed')
return original_replace(path, target)
monkeypatch.setattr(quick_run, '_write_status_details', status); monkeypatch.setattr(Path, 'replace', replace)
original_task_status = store.set_task_status
def task_status(task_id, state, **kwargs):
if failure == 'sql_status' and state is TaskStatus.READY and not count:
count.append(1); raise OSError('task status publication failed')
return original_task_status(task_id, state, **kwargs)
monkeypatch.setattr(store, 'set_task_status', task_status)
with pytest.raises((TaskCenterError, OSError)):service.recover_quick_run(task, confirmed=True)
assert (root / 'uncertain.flag').exists() is (failure != 'sql_status')
reopened = TaskCenterService(store, configure_logging(service._logging.root), quick_run_preparer=preparer)
assert reopened.recover_quick_run(task, confirmed=True).state == 'saved_ok'
assert len([r for r in HistoryRepository(store).page().rows if r['operation'] == '器件替换']) == 2
@@ -0,0 +1,37 @@
import json
from pathlib import Path
import subprocess
import pytest
from dx_productivity.trial_task_center.execution_proof import validate_project_commit_source
@pytest.mark.parametrize('resume_later_job', [False, True])
def test_native_closed_checkpoint_binds_project_marker_and_saved_source(tmp_path, monkeypatch, resume_later_job):
from test_batch_runtime import command_for, worker_runtime
command, project = command_for(tmp_path)
root = Path(command[3])
if resume_later_job:
with (root / 'command.tsv').open('a') as stream:
stream.write(f'task\tnext\t{project}\told\tnew\n')
command = (*command[:-1], 'next')
job = command[-1]
token = 'a' * 32
(root / 'project-session.request').write_text(f'{token}\t{project}\t{job}')
marker = root / ('project-commit-' + token + '.tsv')
def monitor(*args, **kwargs):
marker.write_text(f'{token}\t{project}\tproject_verified\n{job}\tno_match\n')
(root / ('post-state-' + job + '.tsv')).write_text('native final semantic state')
return subprocess.CompletedProcess(command, 0, '', '')
runtime = worker_runtime(tmp_path, monkeypatch, monitor)
with runtime:
runtime.run_worker(command, 5)
checkpoint = json.loads((root / 'project-checkpoint.json').read_text(encoding='utf-8'))
proof = checkpoint['project_commit_source']
assert proof['session_token'] == token and proof['jobs'] == [job]
assert set(proof['readback_files']) == {'post-state-' + job + '.tsv'}
assert proof['native_attempt'] == {'session_id': runtime.session_id, 'generation': runtime.generation,
'attempt_id': runtime.record['active_attempt']}
assert validate_project_commit_source(proof, root)
marker.write_text('changed')
assert not validate_project_commit_source(proof, root)
+44
View File
@@ -0,0 +1,44 @@
from pathlib import Path
import json
from types import SimpleNamespace
from dx_productivity.trial_task_center.project_session import ProjectSessionScope
from test_project_session import toy_worker
def test_staged_worker_stays_open_until_project_commit_and_callback_runs_once(tmp_path):
command = toy_worker(tmp_path)
script = Path(command[2])
script.write_text(script.read_text(encoding='utf-8').replace('saved_ok', 'pending_save').replace('session_verified', 'session_staged'), encoding='utf-8')
commits = []
with ProjectSessionScope(None, enabled=True) as scope:
result = scope.run_step(tmp_path / 'board.prj', command, 3)
assert result.returncode == 0
assert not (tmp_path / 'closed.txt').exists()
scope.on_commit = lambda worker: commits.append(worker.token)
assert scope.run_step(tmp_path / 'board.prj', (*command[:-1], 'job2'), 3).returncode == 0
assert (tmp_path / 'project-session.request').read_text(encoding='utf-8').split('\t')[2] == 'job2'
scope.close()
assert (tmp_path / 'closed.txt').exists()
scope.close()
assert len(commits) == 1
def test_project_close_extends_wait_while_native_commit_keeps_reporting_progress(tmp_path, monkeypatch):
from dx_productivity.trial_task_center import project_session
elapsed = [0.0]
class FinishingCommit:
def is_set(self):return elapsed[0] >= 12
def wait(self, timeout):
elapsed[0] += .1
(tmp_path / 'worker-progress.json').write_text(json.dumps({'sequence': int(elapsed[0]), 'native_progress_frames': 0}))
return self.is_set()
monkeypatch.setattr(project_session, 'time', SimpleNamespace(monotonic=lambda: elapsed[0], sleep=lambda seconds: None))
worker = project_session.ProjectSessionWorker(tmp_path / 'board.prj', tmp_path)
worker._thread = object()
worker._timeout = 1
worker._done = FinishingCommit()
worker._result = SimpleNamespace(returncode=0)
worker.close()
assert elapsed[0] >= 12
assert not worker._abort.is_set()
+27
View File
@@ -0,0 +1,27 @@
import hashlib
import pytest
from test_value_readback import case
from dx_productivity.trial_task_center.value_readback import capture_value_readback, load_value_readback
from dx_productivity.trial_task_center.quick_run_protocol import _encode_vbs_field
@pytest.mark.parametrize('damage',['','foreign_project','changed_marker','wrong_state'])
def test_committed_prefix_readback_is_bound_to_manifest_not_last_status(tmp_path,damage):
row,_=case(tmp_path)
token='a'*32
marker=tmp_path/f'project-commit-{token}.tsv'
marker.write_text(f'{token}\t{_encode_vbs_field(row["project_path"])}\tproject_verified\njob\tsaved_ok\nnext\tno_match\n')
proof={'session_token':token,'project_path':row['project_path'],'jobs':['job','next'],
'outcomes':{'job':'saved_ok','next':'no_match'},
'files':[{'path':str(marker),'sha256':hashlib.sha256(marker.read_bytes()).hexdigest()}]}
with (tmp_path/'status.tsv').open('a') as stream:
stream.write('next\tno_match\tproject_committed\tverified\n')
if damage=='foreign_project':proof['project_path']=str(tmp_path/'other.prj')
elif damage=='changed_marker':marker.write_text('changed')
elif damage=='wrong_state':proof['outcomes']['job']='no_match'
result=capture_value_readback(tmp_path,row,('saved_ok','project_committed','verified'),project_commit=proof)
assert result['value_readback_status']==('rejected' if damage else 'verified')
if not damage:
assert load_value_readback(tmp_path,row,('saved_ok','project_committed','verified'))==result
+403
View File
@@ -0,0 +1,403 @@
from pathlib import Path
import re
import subprocess
import pytest
SOURCE = Path(__file__).resolve().parents[2] / 'integration/dxdesigner795/QuickPartReplace.vbs'
def procedure(name):
source = SOURCE.read_text(encoding='utf-8-sig')
match = re.search(rf'(?ms)^Sub {name}\(.*?^End Sub', source)
assert match, f'missing production procedure {name}'
return match.group()
def run_script(tmp_path, script):
path = tmp_path / 'production-extract.vbs'
path.write_text(script, encoding='ascii')
return subprocess.run((r'C:\Windows\SysWOW64\cscript.exe', '//nologo', str(path)),
capture_output=True, text=True, timeout=10)
def test_production_batch_finish_stages_session_item_before_per_step_save(tmp_path):
result = run_script(tmp_path, '''QP_SessionMode = True
QP_BatchState = "restored_ok"
QPFinishBatchItem
Sub QPStageSessionItem()
WScript.Echo "pending_save"
End Sub
''' + procedure('QPFinishBatchItem'))
assert result.returncode == 0, result.stderr
assert result.stdout.strip() == 'pending_save'
def test_production_baseline_wrapper_preserves_safe_pause_source(tmp_path):
result = run_script(tmp_path, '''On Error Resume Next
QPVerifyExpectedBaselines "board"
WScript.Echo Err.Source
Sub QPCheckExpectedBaselines(ByVal TargetPath)
Err.Raise vbObjectError + 860, "QuickPartReplace.SafeStop", "paused"
End Sub
''' + procedure('QPVerifyExpectedBaselines'))
assert result.returncode == 0, result.stderr
assert result.stdout.strip() == 'QuickPartReplace.SafeStop'
def test_production_project_commit_closes_empty_owned_project_without_save(tmp_path):
result = run_script(tmp_path, '''Set QP_SessionItems = CreateObject("Scripting.Dictionary")
QP_SessionProject = "board.prj"
QPCommitSessionProject
Sub QPCloseBatchProject(ByVal Target, ByVal SaveChanges)
WScript.Echo Target & ":" & CStr(SaveChanges)
End Sub
''' + procedure('QPCommitSessionProject'))
assert result.returncode == 0, result.stderr
assert result.stdout.strip() == 'board.prj:False'
def test_production_pause_acknowledges_only_after_original_proof_and_prefix_close(tmp_path):
script = '''Set QP_Fso = CreateObject("Scripting.FileSystemObject")
Set QP_SessionItems = CreateObject("Scripting.Dictionary")
QP_CurrentJobId = "current"
QP_ExpectedJobId = "current"
QP_MonitorJobId = "initial"
QP_OwnedProject = True
QP_Root = "."
QP_SessionToken = String(32, "a")
QP_SessionProject = "board.prj"
QP_AttemptToken = "monitor"
QP_ProgressSequence = 3
QPAcknowledgePrewritePause
WScript.Echo "unsafe-no-ack"
Sub QPVerifyFailedOriginalSnapshot()
WScript.Echo "original-verified"
End Sub
Function QPReadCommand(ByVal Path)
Set QPReadCommand = CreateObject("Scripting.Dictionary")
End Function
Sub QPWriteVerifiedState(ByRef Command)
WScript.Echo "unchanged-semantic-state"
End Sub
Sub QPCommitSessionProject()
WScript.Echo "prefix-committed-and-closed"
QP_OwnedProject = False
End Sub
Sub QPWriteUtf8(ByVal Path, ByVal Data)
WScript.Echo "durable-pause-marker"
End Sub
Function QPEncode(ByVal Value)
QPEncode = Value
End Function
'''
result = run_script(tmp_path, script + procedure('QPAcknowledgePrewritePause'))
assert result.returncode == 75
assert result.stdout.splitlines()[:4] == ['original-verified', 'unchanged-semantic-state',
'prefix-committed-and-closed', 'durable-pause-marker']
assert result.stdout.splitlines()[4] == 'DX_PREWRITE_PAUSED\tmonitor\tinitial\t' + 'a' * 32 + '\tcurrent\t3'
unsafe = run_script(tmp_path, script.replace('QPAcknowledgePrewritePause\n',
'QP_NativeMutationPossible = True\nQPAcknowledgePrewritePause\n', 1) + procedure('QPAcknowledgePrewritePause'))
assert unsafe.returncode == 0
assert unsafe.stdout.strip() == 'unsafe-no-ack'
def test_production_commit_saves_each_sheet_once_and_reopens_project_once(tmp_path):
# COM is the test boundary; all project commit/close sequencing comes from VBS.
script = '''On Error Resume Next
Set QP_Fso = CreateObject("Scripting.FileSystemObject")
QP_Root = "."
QP_SessionMode = True
QP_SessionProject = "board.prj"
QP_OwnedProjectPath = "board.prj"
QP_OwnedProject = True
QP_ProjectOpen = True
QP_DocCount = 2
QP_Saves = 0
QP_Closes = 0
QP_Opens = 0
QP_Reads = 0
QP_TargetSchematic = ""
QP_TargetSheet = ""
QP_CurrentJobId = ""
QP_SessionToken = String(32, "a")
Set QP_App = New FakeApp
Set QP_SessionItems = CreateObject("Scripting.Dictionary")
Set QP_SessionSheets = CreateObject("Scripting.Dictionary")
Set QP_SessionDocuments = CreateObject("Scripting.Dictionary")
QP_SessionDocuments.Add "A", True
QP_SessionDocuments.Add "B", True
QP_SessionSheets.Add "A:1", "finalA"
QP_SessionSheets.Add "B:1", "finalB"
For I = 1 To 3
Set Entry = CreateObject("Scripting.Dictionary")
Set Command = CreateObject("Scripting.Dictionary")
Command.Add "job_id", "job" & I
Set Records = CreateObject("Scripting.Dictionary")
Set Components = CreateObject("Scripting.Dictionary")
Records.Add "components", Components
Entry.Add "command", Command
Entry.Add "snapshot_path", "snapshot-job" & I
Set Values = CreateObject("Scripting.Dictionary")
Entry.Add "value_states", Values
Entry.Add "schematic", "A"
If I = 3 Then Entry("schematic") = "B"
Entry.Add "sheet", "1"
Entry.Add "final_state", "restored_ok"
QP_SessionItems.Add "job" & I, Entry
Next
QPCommitSessionProject
If Err.Number <> 0 Then
WScript.StdErr.WriteLine Err.Source & ":" & Err.Description
WScript.Quit 1
End If
WScript.Echo "counts=" & QP_Saves & "," & QP_Closes & "," & QP_Opens & "," & QP_Reads
Class FakeDocument
Sub Close(ByVal SaveChanges, ByVal Ignored)
If SaveChanges Then QP_Saves = QP_Saves + 1
QP_DocCount = QP_DocCount - 1
End Sub
End Class
Class FakeDocuments
Property Get Count
Count = QP_DocCount
End Property
Function Item(ByVal Index)
Set Item = New FakeDocument
End Function
End Class
Class FakeProjectData
Function GetProjectFilePath()
GetProjectFilePath = ""
If QP_ProjectOpen Then GetProjectFilePath = "board.prj"
End Function
End Class
Class FakeApp
Function SchematicSheetDocuments()
Set SchematicSheetDocuments = New FakeDocuments
End Function
Function GetProjectData()
Set GetProjectData = New FakeProjectData
End Function
Sub CloseProject()
QP_Closes = QP_Closes + 1
QP_ProjectOpen = False
End Sub
End Class
Sub QPEnsureProject(ByRef App, ByVal Project)
If Not QP_ProjectOpen Then
QP_ProjectOpen = True
QP_Opens = QP_Opens + 1
QP_DocCount = 2
QP_OwnedProject = True
QP_OwnedProjectPath = Project
QP_SessionDocuments.Add "A", True
QP_SessionDocuments.Add "B", True
End If
End Sub
Function QPSheetKey(ByVal Schematic, ByVal Sheet)
QPSheetKey = Schematic & ":" & Sheet
End Function
Function QPCurrentSemanticState()
QP_Reads = QP_Reads + 1
QPCurrentSemanticState = "final" & QP_TargetSchematic
End Function
Sub QPRequireExpectedState(ByVal Expected, ByVal Actual)
If Expected <> Actual Then Err.Raise 1, "Readback", "changed sheet"
End Sub
Function QPProjectPath()
QPProjectPath = "board.prj"
End Function
Function QPCanonicalPath(ByVal Value)
QPCanonicalPath = Value
End Function
Sub QPProgress(ByVal Stage, ByVal Done, ByVal Total)
End Sub
Sub QPLog(ByVal Root, ByVal Level, ByVal Message)
End Sub
Sub QPInvalidatePinIndex()
End Sub
Sub QPInvalidateComponentIndex()
End Sub
Sub QPWriteSemanticState(ByRef Command, ByVal SemanticState)
End Sub
Function QPReadSnapshot(ByVal Path)
Set Result = CreateObject("Scripting.Dictionary")
Set Parts = CreateObject("Scripting.Dictionary")
Result.Add "components", Parts
Set QPReadSnapshot = Result
End Function
Sub QPVerifyRestoredPlans(ByRef Command, ByRef Records, ByRef Plans, ByRef Values)
End Sub
Sub QPWriteValueReadback(ByRef Command, ByRef Records, ByRef Plans, ByVal State)
End Sub
Function QPEncode(ByVal Value)
QPEncode = Value
End Function
Sub QPWriteUtf8(ByVal Path, ByVal Data)
WScript.Echo "marker"
End Sub
Sub QPWriteStatus(ByVal Root, ByVal Job, ByVal State, ByVal Code, ByVal Message)
WScript.Echo Job & ":" & State
End Sub
'''
result = run_script(tmp_path, script + procedure('QPCommitSessionProject') + '\n' + procedure('QPCloseBatchProject'))
assert result.returncode == 0, result.stderr
assert result.stdout.splitlines() == ['marker', 'job1:saved_ok', 'job2:saved_ok', 'job3:saved_ok', 'counts=2,2,1,4']
# A failed reopened sheet readback must not publish the commit marker/status.
changed = script.replace('QPCurrentSemanticState = "final" & QP_TargetSchematic',
'QPCurrentSemanticState = "final" & QP_TargetSchematic\n If QP_Opens > 0 Then QPCurrentSemanticState = "changed"')
failed = run_script(tmp_path, changed + procedure('QPCommitSessionProject') + '\n' + procedure('QPCloseBatchProject'))
assert failed.returncode == 1
assert 'Readback:changed sheet' in failed.stderr
assert 'marker' not in failed.stdout
assert 'saved_ok' not in failed.stdout
@pytest.mark.parametrize('state', ['restored_ok', 'no_match'])
def test_stage_rereads_real_current_command_before_publishing_pending_save(tmp_path, state):
import test_accuracy_vbs_guards as guards
header='task_id\tjob_id\tproject_path\told_part\tnew_part\tvalue_mode\n'
(tmp_path/'command.tsv').write_text(header+'task\tother\tboard.prj\tOLD\tNEW\tpreserve\n')
(tmp_path/'current-command.tsv').write_text(header+'task\tjob\tboard.prj\tOLD\tNEW\tpreserve\n')
(tmp_path/'current-state.tsv').write_text('job_id\tstate\tcode\tmessage\njob\tawaiting_native_replace\t\tdispatched\n')
names=['QPStageSessionItem','QPReadCommand','QPActiveCommandPath','QPReadProjectRuleStates',
'QPReadState','QPIsNextRuleState','QPReadUtf8','QPWriteUtf8','QPWriteStatus','QPAppendAscii','QPSheetKey']
output=guards.run_vbs(tmp_path,names,'''
QP_Root="'''+str(tmp_path)+'''"
Set QP_Fso=CreateObject("Scripting.FileSystemObject")
Set QP_App=New FakeApp
Set QP_SessionItems=CreateObject("Scripting.Dictionary")
Set QP_SessionSheets=CreateObject("Scripting.Dictionary")
QP_ExpectedJobId="job":QP_CurrentJobId="job":QP_BatchMode=True
QP_BatchState="'''+state+'''":QP_SessionToken=String(32,"a")
QPStageSessionItem
WScript.Echo QP_SessionItems("job")("command")("job_id")
WScript.Echo QP_SessionItems("job")("final_state")
WScript.Echo CStr(InStr(QPReadUtf8(QP_Fso.BuildPath(QP_Root,"status.tsv")),"pending_save")>0)
WScript.Echo QP_BatchState
''',prelude='''
Dim QP_Root,QP_Fso,QP_App,QP_SessionItems,QP_SessionSheets,QP_ExpectedJobId,QP_CurrentJobId
Dim QP_BatchMode,QP_BatchState,QP_SessionToken
Class FakeApp
Property Get ActiveView()
Set ActiveView=Me
End Property
Function GetName(ByVal Ignored)
GetName="MAIN"
End Function
Property Get Block()
Set Block=Me
End Property
Property Get SheetNum()
SheetNum="1"
End Property
End Class
Function QPProjectPath()
QPProjectPath="board.prj"
End Function
Function QPCanonicalPath(ByVal Value)
QPCanonicalPath=Value
End Function
Function QPValueMode(ByRef Command)
QPValueMode=Command("value_mode")
End Function
Function QPReadSnapshot(ByVal Path)
Dim R,C:Set R=CreateObject("Scripting.Dictionary"):Set C=CreateObject("Scripting.Dictionary")
R.Add "components",C:Set QPReadSnapshot=R
End Function
Function QPJobSnapshotPath(ByVal Job)
QPJobSnapshotPath="unused-snapshot"
End Function
Sub QPRequireCommandSnapshot(ByRef Command,ByVal Path)
End Sub
Sub QPVerifyUntouched(ByRef Records)
End Sub
Function QPCurrentSemanticState()
QPCurrentSemanticState="verified-memory"
End Function
''')
assert output.splitlines()==['job',state,'True',state]
assert (tmp_path/'project-session-result.tsv').read_text(encoding='utf-8-sig').split('\t')[2]=='pending_save'
@pytest.mark.parametrize('continuation', ['close','next'])
def test_pause_during_staged_checkpoint_finishes_prefix_or_pauses_next_prewrite(tmp_path, continuation):
import test_accuracy_vbs_guards as guards
token='a'*32
(tmp_path/'monitor-monitor.stop').write_text('monitor')
(tmp_path/'project-session-next.tsv').write_text(token+'\t'+continuation)
(tmp_path/'batch.request').write_text('next')
output=guards.run_vbs(tmp_path,['QPSessionNext','QPRequireSessionCheckpoint','QPProgress','QPReadUtf8','QPSheetKey'],'''
QP_Root="'''+str(tmp_path)+'''"
Set QP_Fso=CreateObject("Scripting.FileSystemObject")
Set QP_App=New App
Set QP_SessionItems=CreateObject("Scripting.Dictionary"):QP_SessionItems.Add "previous",True
Set QP_SessionDocuments=CreateObject("Scripting.Dictionary"):QP_SessionDocuments.Add "MAIN:1",True
Set QP_SessionSheets=CreateObject("Scripting.Dictionary"):QP_SessionSheets.Add QPSheetKey("MAIN","1"),"original"
QP_SessionMode=True:QP_SessionWaiting=True:QP_SessionToken="'''+token+'''"
QP_SessionProject="board":QP_OwnedProject=True:QP_OwnedProjectPath="board"
QP_CheckpointSchematic="MAIN":QP_CheckpointSheet="1"
QP_ExpectedJobId="previous":QP_CurrentJobId="previous":QP_MonitorJobId="initial"
QP_AttemptToken="monitor":QP_ProgressSequence=0
QPSessionNext
If QP_SessionMode Then
On Error Resume Next
QPProgress "before_native",0,1
WScript.Echo "next_prewrite_stop=" & CStr(Err.Source="QuickPartReplace.SafeStop")
End If
''',prelude='''
Dim QP_Root,QP_Fso,QP_App,QP_SessionItems,QP_SessionDocuments,QP_SessionSheets
Dim QP_SessionMode,QP_SessionWaiting,QP_SessionToken,QP_SessionProject,QP_OwnedProject,QP_OwnedProjectPath
Dim QP_CheckpointSchematic,QP_CheckpointSheet,QP_ExpectedJobId,QP_CurrentJobId,QP_MonitorJobId
Dim QP_AttemptToken,QP_ProgressSequence
Class App
Function SchematicSheetDocuments()
Set SchematicSheetDocuments=Me
End Function
Property Get Count()
Count=1
End Property
Property Get ActiveView()
Set ActiveView=Me
End Property
Function GetName(ByVal Flags)
GetName="MAIN"
End Function
Property Get Block()
Set Block=Me
End Property
Property Get SheetNum()
SheetNum="1"
End Property
End Class
Function QPProjectPath()
QPProjectPath="board"
End Function
Function QPCanonicalPath(ByVal Value)
QPCanonicalPath=Value
End Function
Function QPReadContinuation(ByVal Path)
QPReadContinuation=QPReadUtf8(Path)
End Function
Function QPCurrentSemanticState()
QPProgress "capture_semantic_state",1,104
QPCurrentSemanticState="original"
End Function
Sub QPRequireExpectedState(ByVal Expected,ByVal Actual)
If Expected<>Actual Then Err.Raise 1,"checkpoint","changed"
End Sub
Sub QPCommitSessionProject()
WScript.Echo "prefix_commit"
End Sub
Function QPConfirmedProjectPath()
QPConfirmedProjectPath="board"
End Function
Sub QPEnsureProject(ByRef App,ByVal Path)
End Sub
''')
lines=output.splitlines()
semantic=[line for line in lines if '\tcapture_semantic_state\t' in line]
assert len(semantic)==1 and semantic[0].split('\t')[-2:]==['0','previous']
assert ('prefix_commit' in lines) if continuation=='close' else ('next_prewrite_stop=True' in lines)
@@ -0,0 +1,65 @@
from pathlib import Path
from threading import Event
from types import SimpleNamespace
import pytest
from dx_productivity.trial_task_center.errors import TaskCenterError
from dx_productivity.trial_task_center.task_queue import execute_task_queue
from test_task_queue import Service
class ReleasedRuntime:
def __init__(self,root,released=True):
self.root=root
self.stop_event=None
self.cad=None
self.released=released
self.release_calls=[]
def bind_logs(self,*args):
pass
def release_failed_project(self,error,evidence_root):
self.release_calls.append(error.code)
return self.released
@pytest.mark.parametrize('released',[False,True])
def test_uncertain_project_only_holds_independent_tasks_until_writers_exit(tmp_path,released):
service=Service(tmp_path,{'A':TaskCenterError('quick_run_uncertain','unknown saved result')})
runtime=ReleasedRuntime(tmp_path,released)
result=execute_task_queue(service,('A','B'),stop_event=Event(),runtime=runtime)
assert result.entries[0].state=='blocked'
assert result.paused is (not released)
assert service.calls==([('start','A'),('start','B'),('end','B')] if released else [('start','A')])
assert runtime.release_calls==['quick_run_uncertain']
def test_local_failure_cleanup_does_not_become_user_batch_pause(tmp_path):
from dx_productivity.trial_task_center.stop_control import ProjectStopControl,StopControl
parent=StopControl()
local=ProjectStopControl(parent)
local.request_failure_cleanup(tmp_path,'injected project failure')
assert local.is_set() and local.force_event.is_set()
assert not parent.is_set() and not parent.force_event.is_set()
next_project=ProjectStopControl(parent)
assert not next_project.is_set()
next_project.set()
assert parent.is_set()
def test_resolution_failure_release_keeps_the_parent_user_intent_unset(tmp_path):
from dx_productivity.trial_task_center.stop_control import ProjectStopControl,StopControl,wait_for_resolution
parent=StopControl()
local=ProjectStopControl(parent)
wait_for_resolution(local,tmp_path,'injected failure',resolved=lambda:True,release_on_failure=True)
assert local.is_set() and not parent.is_set()
def test_isolated_project_is_not_replayed_on_queue_resume(tmp_path):
service=Service(tmp_path,{'A':TaskCenterError('quick_run_uncertain','unknown')})
runtime=ReleasedRuntime(tmp_path)
first=execute_task_queue(service,('A','B'),stop_event=Event(),runtime=runtime)
before=list(service.calls)
resumed=execute_task_queue(service,('A','B'),stop_event=Event(),runtime=runtime,queue_id=first.queue_id)
assert service.calls==before
assert resumed.entries[0].state=='blocked'
+7 -3
View File
@@ -50,9 +50,13 @@ def test_changed_project_after_controlled_failure_blocks_remaining_uid(component
project = service.projects(task_id)[0].path
outcomes[(task_id, ('R1',))] = 'batch_failed'
outcomes['after_call'] = lambda count: project.write_text('unexpected source mutation') if count == 1 else None
result = execute_task_queue(service, (task_id,), stop_event=Event())
assert result.paused
assert calls == [(task_id, ('R1',))]
ids = tuple(task.task_id for task in tasks)
result = execute_task_queue(service, ids, stop_event=Event())
assert not result.paused and [entry.state for entry in result.entries] == ['blocked', 'completed']
assert calls == [(task_id, ('R1',)), (ids[1], ('R1',)), (ids[1], ('R2',))]
assert service.has_interrupted_quick_run(task_id)
execute_task_queue(service, ids, stop_event=Event(), queue_id=result.queue_id)
assert len(calls) == 3
def test_tampered_failure_health_denies_resume(component_case):
+550
View File
@@ -0,0 +1,550 @@
"""Reference designators remain usable when native UID metadata is broken."""
from copy import deepcopy
from types import SimpleNamespace
import pytest
from dx_productivity.trial_task_center import native_inventory_protocol as inventory, native_netlist
from dx_productivity.trial_task_center.target_plan import ComponentTarget, TargetPlanRow, compile_target_rules
from test_native_ascii_inventory import report, effective_report
@pytest.mark.parametrize('damage', ['invalid', 'duplicate', 'changed_between_reports'])
def test_inventory_joins_real_parts_by_refdes_when_uid_is_unusable(tmp_path, damage):
hier,flat=report(tmp_path),effective_report(tmp_path,part='OLD')
if damage=='invalid':
hier.write_text(hier.read_text().replace('$27I100','broken'))
flat.write_text(flat.read_text().replace('$27I100','broken'))
elif damage=='duplicate':
hier.write_text(hier.read_text().replace('$28I200','$27I100'))
flat.write_text(flat.read_text().replace('$28I200','$27I100'))
else:
flat.write_text(flat.read_text().replace('$27I100','$99I999'))
result=inventory.read_ascii_discovery(tmp_path/'board.prj',hier,('OLD',),effective_report_path=flat)
assert {(row.refdes,row.old_part_number,row.sheet_number) for row in result.components}=={
('R1','OLD','MAIN'),('R2','OLD','AUX')}
assert len({row.key for row in result.components})==2
def test_plan_target_identity_is_refdes_based_without_rewriting_legacy_uid(tmp_path):
old=ComponentTarget(tmp_path/'board.prj','MAIN','1','legacy-u1','R1','OLD','1K')
new=ComponentTarget(tmp_path/'board.prj','MAIN','1','changed-u1','R1','OLD','1K')
assert old.key==new.key
assert old.uid=='legacy-u1'
with pytest.raises(Exception,match='more than once|重复'):
compile_target_rules((TargetPlanRow(old,True,'NEW'),TargetPlanRow(new,True,'NEW')))
def test_blank_uid_can_be_represented_by_a_reference_target(tmp_path):
target=ComponentTarget(tmp_path/'board.prj','MAIN','1','','R1','OLD','1K')
assert target.uid=='@REFDES|R1'
def graph(uid):
return {'status':'success','complete':True,'graph':{
'components':{uid:{'refdes':'R1','attributes':{'Part Number':'OLD'},
'symbol':'res.1 type Primitive ForwardPCB',
'pins':{'p1':{'number':'1','nets':['GND']}}}},
'nets':[{'name':'GND','endpoints':[[uid,'p1']]}]}}
def test_uid_change_alone_does_not_change_netlist_acceptance():
before,after=graph('$1I1'),graph('$2I2')
result=native_netlist.compare_netlists(before,after)
assert result['status']=='verified',result
assert '$1I1' in before['graph']['components'] # comparisons do not mutate evidence
def test_reference_mapping_does_not_hide_real_pin_net_changes():
before,after=graph('$1I1'),graph('$2I2')
after['graph']['components']['$2I2']['pins']['p1']['nets']=['VCC']
after['graph']['nets']=[{'name':'VCC','endpoints':[['$2I2','p1']]}]
result=native_netlist.compare_netlists(before,after)
assert result['status']=='different'
def test_native_readback_can_reassociate_changed_uid_by_unique_refdes(monkeypatch):
from dx_productivity.trial_task_center import native_helper
components=[SimpleNamespace(UID='new-uid',Refdes='R1')]
values={'Part Number':'NEW','Part Name':'LIB:NEW','Part Label':'LABEL'}
monkeypatch.setattr(native_helper,'attribute_value',lambda component,name,**options:values.get(name,''))
records=[['C','old-uid','R1','0','0','0','1','']]
assert native_helper.replacement_matches(components,records,'NEW',True,
library_row=['','NEW','','','LIB:NEW','LABEL'])==['old-uid']
def test_native_snapshot_scope_binds_legacy_keys_by_reference_and_resets(tmp_path):
from dx_productivity.trial_task_center import native_helper
snapshot=tmp_path/'snapshot.tsv'
snapshot.write_text('M\tjob\tproject.prj\tOLD\tNEW\tMAIN:1\n'
'C\tlegacy\tR1\t0\t0\t0\t1\t\n')
component=SimpleNamespace(UID='changed',Refdes='R1')
with native_helper.snapshot_identity_scope(snapshot):
assert native_helper._component_uid(component)=='legacy'
assert native_helper._component_uid(component)=='changed'
def test_complete_ascii_graph_does_not_require_numeric_uid(tmp_path):
from test_native_cli_netlist import ascii_report
ascii_path,keyin=tmp_path/'flat.txt',tmp_path/'design.kyn'
ascii_path.write_text(ascii_report().replace('$1I1','broken-uid'))
keyin.write_text('%net\n%page=board\n\\N1\\ \\R1\\-\\1\\\n%Part\n\\OLD\\ \\R1\\\n')
result=native_netlist.read_graph(ascii_path,keyin)
assert {row['refdes'] for row in result['components'].values()}=={'R1'}
def test_vbs_selector_can_resolve_a_changed_uid_by_reference(tmp_path):
import test_accuracy_vbs_guards as guards
output=guards.run_vbs(tmp_path,['QPComponentIdentityKey'],'''
Dim Comp
Set Comp=New FakeComponent
Set QP_ReferenceAliases=CreateObject("Scripting.Dictionary")
QP_ReferenceAliases.Add "main:1|r1","legacy-id"
WScript.Echo QPComponentIdentityKey(Comp)
''',prelude='''
Dim QP_ReferenceAliases
Class FakeComponent
Public UID,Refdes
Private Sub Class_Initialize()
UID="changed-id":Refdes="R1"
End Sub
End Class
Function QPSheetIdentity()
QPSheetIdentity="MAIN:1"
End Function
''')
assert output=='legacy-id'
def test_netlist_plan_can_bind_reference_selector_to_original_graph_uid():
from dx_productivity.trial_task_center.netlist_verification import _reference_plan
plan={'rows':[{'enabled':True,'target':{'uid':'@REFDES|R1','refdes':'R1',
'schematic_name':'MAIN','sheet_number':'1','old_value':'1K'},'new_part_number':'NEW'}]}
rebound,library=_reference_plan(plan,graph('$1I1'),{'@REFDES|R1':{
'refdes':'R1','schematic_name':'MAIN','sheet_number':'1','attributes':{'Part Number':'NEW'}}})
assert rebound['rows'][0]['target']['uid']=='$1I1'
assert '$1I1' in library
assert plan['rows'][0]['target']['uid']=='@REFDES|R1'
def test_reused_uid_cannot_override_a_known_reference_mismatch():
from dx_productivity.trial_task_center.netlist_verification import _reference_plan
plan={'rows':[{'enabled':True,'target':{'uid':'$1I1','refdes':'R2','old_value':'1K'},'new_part_number':'NEW'}]}
with pytest.raises(ValueError,match='位号'):
_reference_plan(plan,graph('$1I1'),{})
def test_vbs_unselected_duplicate_uids_get_distinct_reference_keys(tmp_path):
import test_accuracy_vbs_guards as guards
output=guards.run_vbs(tmp_path,['QPComponentIdentityKey','QPBindSheetReferences'],'''
Dim Rows,A,B
Set Rows=CreateObject("Scripting.Dictionary")
Set A=New FakeComponent:A.UID="duplicate":A.Refdes="R2"
Set B=New FakeComponent:B.UID="duplicate":B.Refdes="R3"
Rows.Add A,True:Rows.Add B,True
QPBindSheetReferences Rows
WScript.Echo QPComponentIdentityKey(A)
WScript.Echo QPComponentIdentityKey(B)
''',prelude='''
Dim QP_ReferenceAliases
Class FakeComponent
Public UID,Refdes
End Class
Function QPSheetIdentity()
QPSheetIdentity="MAIN:1"
End Function
''')
assert output.splitlines()==['@REFDES|R2','@REFDES|R3']
def test_inventory_disambiguates_same_reference_by_sheet(tmp_path):
hier, flat = report(tmp_path), effective_report(tmp_path, part='OLD')
hier.write_text(hier.read_text().replace('Ref Designator R2', 'Ref Designator R1'))
flat.write_text(flat.read_text().replace('ref des R1\tpage ', 'ref des R1\tpage MAIN')
.replace('ref des R2\tpage ', 'ref des R1\tpage AUX')
.replace('Ref Designator R2', 'Ref Designator R1'))
result = inventory.read_ascii_discovery(tmp_path/'board.prj', hier, ('OLD',), effective_report_path=flat)
assert {(row.refdes, row.sheet_number) for row in result.components} == {('R1','MAIN'),('R1','AUX')}
def test_inventory_groups_legal_slots_as_one_physical_target(tmp_path):
hier, flat = report(tmp_path), effective_report(tmp_path, part='OLD')
text = hier.read_text()
first = text[text.index(' Symbol $27I100'):text.index(' Symbol IN_PORT')]
hier.write_text(text.replace(' Symbol IN_PORT', first.replace('$27I100','$27I101').replace('res.1','res.2')+' Symbol IN_PORT'))
text = flat.read_text().replace('\tPath \n', '\tPath \n\tSymbol name res.1 type Primitive ForwardPCB\n')
first = text[text.index(' Symbol $27I100'):text.index(' Symbol $28I200')]
flat.write_text(text+first.replace('$27I100','$27I101').replace('res.1','res.2'))
result = inventory.read_ascii_discovery(tmp_path/'board.prj', hier, ('OLD',), effective_report_path=flat)
assert [(row.refdes,row.sheet_number) for row in result.components] == [('R1','MAIN'),('R2','AUX')]
@pytest.mark.parametrize('uid', ['broken', '$27I100'])
def test_hierarchical_inventory_joins_child_by_reference_despite_uid_damage(tmp_path, uid):
hier, flat = report(tmp_path), effective_report(tmp_path, part='OLD')
text = hier.read_text().replace('Block disconnected \tRoot', 'Block child ')
text = text.replace('Block board \tRoot\n', 'Block board \tRoot\n'
'\tChild child.1\tinstance name $9I2\tview unrecognized\n Symbol $9I2\tpage MAIN\n')
hier.write_text(text.replace('$28I200',uid))
flat.write_text(flat.read_text().replace('$28I200',uid).replace(
'ref des R2\tpage \n\tPath \n', 'ref des R2\tpage AUX\n\tPath $9I2\n')
.replace('Prop Ref Designator R2','Prop Ref Designator 2R2')
+' Symbol IN_PORT\tref des \tpage \n\tPath \n\tSymbol name input.1 type \n')
result = inventory.read_ascii_discovery(tmp_path/'board.prj', hier, ('OLD',), expected_root='board', effective_report_path=flat)
assert {(row.refdes,row.schematic_name,row.sheet_number) for row in result.components} == {('R1','board','MAIN'),('2R2','child','AUX')}
def test_hierarchy_graph_coverage_uses_reference_when_raw_uid_repeats(tmp_path):
from test_native_cli_netlist import ascii_report
text = ascii_report()
text = text.replace('\t\tPath root\n', '\t\tPath root\n\t\tConn $1I1 pin B page \n\t\tPath root\n')
second = text[text.index(' Symbol '):].replace('ref des R1','ref des R2').replace('Ref Designator R1','Ref Designator R2').replace('Pin A','Pin B').replace('num 1','num 2')
flat, kyn, hier = tmp_path/'flat.txt',tmp_path/'g.kyn',tmp_path/'hier.txt'
flat.write_text(text+second)
kyn.write_text('%net\n\\N1\\ \\R1\\-\\1\\ \\R2\\-\\2\\\n%Part\n\\OLD\\ \\R1\\ \\R2\\\n')
header = ascii_report().split('Config ')[0].replace('FLAT BLOCKS','HIER BLOCKS')
hier.write_text(header+'Block board \tRoot\n Symbol $1I1\tpage 1\n\tProp Ref Designator R1\n'
+' Symbol $1I1\tpage 1\n\tProp Ref Designator R2\n')
parsed = native_netlist.read_graph(flat,kyn,hierarchy_path=hier,root_block='board')
assert {row['source_uid'] for row in parsed['components'].values()} == {'$1I1'}
assert {row['refdes'] for row in parsed['components'].values()} == {'R1','R2'}
def test_reference_graph_distinguishes_same_ref_on_different_sheets():
before = graph('a')
before['graph']['components']['a'].update(source_page='1',symbol='res.1 type Primitive ForwardPCB')
second = deepcopy(before['graph']['components']['a'])
second['source_page'] = '2'
before['graph']['components']['b'] = second
before['graph']['nets'][0]['endpoints'].append(['b','p1'])
after = deepcopy(before)
after['graph']['components']['new-a'] = after['graph']['components'].pop('a')
after['graph']['components']['new-b'] = after['graph']['components'].pop('b')
after['graph']['nets'][0]['endpoints'] = [['new-a','p1'],['new-b','p1']]
assert native_netlist.compare_netlists(before,after)['verified']
assert native_netlist.reference_targets(after['graph'],'R1',sheet='2') == ('new-b',)
def test_declared_reference_changes_expand_to_all_legal_symbol_units():
before = graph('a')
before['graph']['components']['a']['symbol'] = 'old.1 type Primitive ForwardPCB'
second = deepcopy(before['graph']['components']['a'])
second['symbol'] = 'old.2 type Primitive ForwardPCB'
second['pins']['p1']['number'] = '2'
before['graph']['components']['b'] = second
before['graph']['nets'][0]['endpoints'].append(['b','p1'])
after = deepcopy(before)
for row in after['graph']['components'].values():
row['attributes']['Part Number'] = 'NEW'
result = native_netlist.compare_netlists(before,after,expected_changes={'@REFDES|R1':{'Part Number':'NEW'}})
assert result['verified'],result
after['graph']['components']['b']['attributes']['Part Number'] = 'WRONG'
assert not native_netlist.compare_netlists(before,after,expected_changes={'@REFDES|R1':{'Part Number':'NEW'}})['verified']
def test_native_inventory_and_reciprocal_graph_accept_missing_uid(tmp_path):
from test_native_cli_netlist import ascii_report
hier, flat = report(tmp_path), effective_report(tmp_path, part='OLD')
hier.write_text(hier.read_text().replace('$27I100',''))
flat.write_text(flat.read_text().replace('$27I100',''))
result = inventory.read_ascii_discovery(tmp_path/'board.prj',hier,('OLD',),effective_report_path=flat)
assert {row.refdes for row in result.components} == {'R1','R2'}
graph_path, keyin = tmp_path/'graph.txt',tmp_path/'graph.kyn'
graph_path.write_text(ascii_report().replace('$1I1',''))
keyin.write_text('%net\n\\N1\\ \\R1\\-\\1\\\n%Part\n\\OLD\\ \\R1\\\n')
parsed = native_netlist.read_graph(graph_path,keyin)
assert [row['source_uid'] for row in parsed['components'].values()] == ['']
@pytest.mark.parametrize('damage', ['missing_value', 'missing_reference'])
def test_hierarchical_inventory_keeps_required_observation_checks(tmp_path, damage):
hier, flat = report(tmp_path), effective_report(tmp_path, part='OLD')
text = hier.read_text().replace('Block disconnected \tRoot', 'Block child ')
text = text.replace('Block board \tRoot\n', 'Block board \tRoot\n'
'\tChild child.1\tinstance name $9I2\tview unrecognized\n Symbol $9I2\tpage MAIN\n')
hier.write_text(text.replace('\tProp Ref Designator R2\n','') if damage == 'missing_reference' else text)
text = flat.read_text()
if damage == 'missing_value':
text = text.replace('\tProp Value 20K\n','')
flat.write_text(text.replace('ref des R2\tpage \n\tPath \n',
'ref des R2\tpage \n\tPath $9I2\n')
+' Symbol IN_PORT\tref des \tpage \n\tPath \n\tSymbol name input.1 type \n')
with pytest.raises(inventory.NativeInventoryUnsupported):
inventory.read_ascii_discovery(tmp_path/'board.prj',hier,('OLD',),expected_root='board',effective_report_path=flat)
def test_six_heterogeneous_units_with_shared_slot_number_compare_by_physical_pins():
from pathlib import Path
fixtures = Path(__file__).parent/'fixtures/native-complex'
parsed = native_netlist.read_graph(fixtures/'six-physical-gates.txt',fixtures/'six-physical-gates.kyn')
result = {'status':'success','complete':True,'graph':parsed}
assert native_netlist.compare_netlists(result,result)['verified']
@pytest.mark.parametrize('field,actual,wanted', [('source_page','2','1'),('source_schematic','OTHER','MAIN')])
def test_reused_uid_cannot_override_known_scope_mismatch(field,actual,wanted):
from dx_productivity.trial_task_center.netlist_verification import _reference_plan
before=graph('$1I1');before['graph']['components']['$1I1'][field]=actual
target={'uid':'$1I1','refdes':'R1', 'sheet_number' if field=='source_page' else 'schematic_name':wanted}
with pytest.raises(ValueError):
_reference_plan({'rows':[{'enabled':True,'target':target}]},before,{})
def test_snapshot_scope_disambiguates_unselected_raw_uid_collisions(tmp_path):
from dx_productivity.trial_task_center import native_helper as helper
path=tmp_path/'snapshot.tsv'
path.write_text('C\tselected\tR1\t0\t0\t0\t1\t\nU\tduplicate\tR2\tOTHER\tx\tn\tl\nU\tduplicate\tR3\tOTHER\tx\tn\tl\n')
with helper.snapshot_identity_scope(path):
identities=[helper._component_uid(SimpleNamespace(UID='duplicate',Refdes=ref)) for ref in ('R2','R3')]
assert identities==['@REFDES|R2','@REFDES|R3']
def unit(uid,x,part='NEW'):
attributes={'Part Number':part,'Part Name':'LIB:NEW','Part Label':'LABEL','Value':'1K'}
component=SimpleNamespace(UID=uid,Refdes='U1',Orientation=0,Scale=1,
GetLocation=lambda:SimpleNamespace(X=x,Y=0),
FindAttribute=lambda name:SimpleNamespace(Value=attributes[name],InstanceValue='') if name in attributes else None)
component._oleobj_=component
return component
def test_multiunit_readback_keeps_original_unit_keys_after_uid_churn(tmp_path):
from dx_productivity.trial_task_center import native_helper as helper
path=tmp_path/'snapshot.tsv'
path.write_text('C\told1\tU1\t0\t0\t0\t1\t\nC\told2\tU1\t100\t0\t0\t1\t\n')
records=[['C','old1','U1','0','0','0','1',''],['C','old2','U1','100','0','0','1','']]
with helper.snapshot_identity_scope(path):
current=[unit('new2',100),unit('new1',0)]
assert [helper._component_uid(component) for component in current]==['old2','old1']
assert helper.replacement_matches(current,records,'NEW',True,library_row=['','NEW','','','LIB:NEW','LABEL'])==['old1','old2']
assert helper.replacement_matches([unit('new1',0),unit('new2',100,'WRONG')],records,'NEW',True) is None
with pytest.raises(ValueError):
helper.replacement_matches([unit('new1',0),unit('new2',0)],records,'NEW',True)
def test_vbs_unit_aliases_survive_uid_changes_and_keep_physical_selector(tmp_path):
import test_accuracy_vbs_guards as guards
output=guards.run_vbs(tmp_path,['QPCommandTargetKey'],'''
Dim Rows,A,B,Selectors
Set Rows=CreateObject("Scripting.Dictionary")
Set A=New FakeComponent:A.UID="old1":A.X=0
Set B=New FakeComponent:B.UID="old2":B.X=100
Rows.Add A,True:Rows.Add B,True
QPBindSheetReferences Rows
A.UID="changed1":B.UID="changed2"
Set Selectors=CreateObject("Scripting.Dictionary")
Selectors.Add "@REFDES|U1",Array("U1","1K")
WScript.Echo QPComponentIdentityKey(A)
WScript.Echo QPComponentIdentityKey(B)
WScript.Echo QPCommandTargetKey(B,Selectors)
''',prelude='''
Class FakePoint
Public X,Y
End Class
Class FakeComponent
Public UID,X,Refdes,Orientation,Scale
Private Sub Class_Initialize()
Refdes="U1":Orientation=0:Scale=1
End Sub
Public Function GetLocation()
Dim P:Set P=New FakePoint:P.X=X:P.Y=0:Set GetLocation=P
End Function
End Class
Function QPSheetIdentity()
QPSheetIdentity="MAIN:1"
End Function
''')
assert output.splitlines()==['old1','old2','@REFDES|U1']
def test_multiunit_value_evidence_covers_physical_selector_without_dropping_units(tmp_path):
from test_value_readback import case
from dx_productivity.trial_task_center.value_readback import capture_value_readback
from dx_productivity.trial_task_center.quick_run_protocol import _encode_vbs_field as enc
row, raw=case(tmp_path)
row['target_selectors']='@REFDES|U1\x1fU1\x1f10nF'
snapshot='C\tu1\tU1\t0\t0\t0\t1\tNO_PINS\r\nC\tu2\tU1\t100\t0\t0\t1\tNO_PINS\r\n'
(tmp_path/'snapshot-job.tsv').write_bytes(snapshot.encode())
header=['V','2','task','job','saved_ok',row['project_path'],'MAIN','1','OLD','NEW','custom','10nF','1','CELL',row['target_selectors'],snapshot]
records=[header,['C','u1','U1','NEW','10nF','1','CELL'],['C','u2','U1','NEW','10nF','1','CELL']]
raw.write_text('\n'.join('\t'.join(map(enc,record)) for record in records)+'\n',encoding='utf8')
result=capture_value_readback(tmp_path,row,('saved_ok','batch_closed','verified'))
assert result['value_readback_status']=='verified',result
assert {item['uid'] for item in result['value_readback']}=={'u1','u2'}
raw.write_text('\n'.join('\t'.join(map(enc,record)) for record in records[:-1])+'\n',encoding='utf8')
assert capture_value_readback(tmp_path,row,('saved_ok','batch_closed','verified'))['value_readback_status']=='rejected'
def test_real_vbs_capture_records_all_units_of_one_physical_selector(tmp_path):
import test_accuracy_vbs_guards as guards
names=guards.UNTOUCHED_ROUTINES+['QPCaptureAndSelect','QPSnapshotComponent','QPCommandTargets',
'QPBooleanText','QPLogField','QPTargetCell','QPRequireUniqueAttributeNames','QPAttributeObservations']
path=tmp_path/'units.tsv'
output=guards.run_vbs(tmp_path,names,'''
Dim A,B,Attr,Pin,Command,Records,Comp
Set QP_App=New FakeApp
Set A=New FullComponent:A.UID="u1":A.Refdes="U1":A.X=0
Set B=New FullComponent:B.UID="u2":B.Refdes="U1":B.X=100
For Each Comp In Array(A,B)
Set Attr=New FullAttribute:Attr.Name="Part Number":Attr.Value="OLD":Attr.InstanceValue="OLD"
Comp.Attributes.Add Attr,True
Set Attr=New FullAttribute:Attr.Name="Value":Attr.Value="1K":Attr.InstanceValue="1K"
Comp.Attributes.Add Attr,True
QP_App.ActiveView.Components.Add Comp,True
Set Pin=New FakePin:Set Pin.Component=Comp:Pin.Number="1":Pin.X=Comp.X:Pin.Y=0
Set Pin.Connection=Nothing:QP_App.ActiveView.Pins.Add Pin,True
Next
Set Command=CreateObject("Scripting.Dictionary")
Command.Add "job_id","test":Command.Add "old_part_number","OLD":Command.Add "new_part_number","NEW"
Command.Add "schematic_name","MAIN":Command.Add "sheet_number","1"
Command.Add "target_scope","targets":Command.Add "target_selectors",QPEncode("@REFDES|U1" & Chr(31) & "U1" & Chr(31) & "1K")
QPCaptureAndSelect Command,"'''+str(path)+'''"
Set Records=QPReadSnapshot("'''+str(path)+'''")
WScript.Echo Records("components").Count
WScript.Echo CapturedState
''',prelude=guards.LOG_STUB+guards.FULL_FAKE+'''
Dim CapturedState
Sub QPBuildComponentIndex(ByRef Components)
End Sub
Function QPValueMode(ByRef Command)
QPValueMode="preserve"
End Function
Function QPCustomValue(ByRef Command)
QPCustomValue=""
End Function
Function QPSyncNc(ByRef Command)
QPSyncNc=False
End Function
Sub QPWriteStatus(ByVal Root,ByVal Job,ByVal State,ByVal Code,ByVal Message)
CapturedState=State
End Sub
''')
assert output.splitlines()==['2','awaiting_native_replace']
def test_fresh_vbs_recovery_restores_interleaved_unselected_unit_aliases(tmp_path):
import test_accuracy_vbs_guards as guards
from dx_productivity.trial_task_center.quick_run_protocol import _encode_vbs_field as enc
path=tmp_path/'recovery.tsv'
text='M\tC:\\board.prj\tOLD\tNEW\tMAIN:1\r\nC\tselected\tR1\t0\t0\t0\t1\t\r\nA\tselected\tValue\t1\t1\t0\t0\t0\t0\t10\t0\t1\r\n'
for uid,x in [('unit1','0'),('unit2','100')]:
state=f'C\t{uid}\tU1\t{x}\t0\t0\t1\tNO_PINS\r\n'
text+=f'U\t{uid}\tU1\tOTHER\tv\tn\tl\r\nG\t{uid}\t{enc(state)}\r\n'
path.write_bytes(text.encode())
output=guards.run_vbs(tmp_path,['QPReadSnapshot','QPReadUtf8'],'''
Dim QP_Fso,Records,A,B
Set QP_Fso=CreateObject("Scripting.FileSystemObject")
Set Records=QPReadSnapshot("'''+str(path)+'''")
Set A=New Unit:A.X=0:Set B=New Unit:B.X=100
WScript.Echo QPComponentIdentityKey(A)
WScript.Echo QPComponentIdentityKey(B)
''',prelude='''
Class Point
Public X,Y
End Class
Class Unit
Public X
Public Property Get UID()
UID="changed"
End Property
Public Property Get Refdes()
Refdes="U1"
End Property
Public Function GetLocation()
Dim P:Set P=New Point:P.X=X:P.Y=0:Set GetLocation=P
End Function
Public Property Get Orientation()
Orientation=0
End Property
Public Property Get Scale()
Scale=1
End Property
End Class
Function QPSheetIdentity()
QPSheetIdentity="MAIN:1"
End Function
''')
assert output.splitlines()==['unit1','unit2']
def test_all_match_pre_submit_rejects_reused_uid_with_changed_ref(monkeypatch):
from dx_productivity.trial_task_center import native_helper as helper
component=unit('original',0,'OLD');component.Refdes='R2'
app=SimpleNamespace(ActiveView=SimpleNamespace(Query=lambda *args:[component]))
monkeypatch.setattr(helper,'check_active_context',lambda *args:None)
with pytest.raises(ValueError,match='reference|Reference'):
helper.check_context(app,SimpleNamespace(IVdComp=lambda value:value),
['M','board','OLD','NEW','MAIN:1','preserve'],[['C','original','R1','0','0','0','1','']],require_selection=False)
def test_vbs_snapshot_does_not_alias_distinct_blank_reference_annotations(tmp_path):
import test_accuracy_vbs_guards as guards
from dx_productivity.trial_task_center.quick_run_protocol import _encode_vbs_field as enc
path=tmp_path/'annotations.tsv'
text='M\tC:\\board.prj\tOLD\tNEW\tMAIN:1\r\nC\tselected\tR1\t0\t0\t0\t1\t\r\nA\tselected\tValue\t1\t1\t0\t0\t0\t0\t10\t0\t1\r\n'
for uid in ('annotation1','annotation2'):
state=f'C\t{uid}\t\t0\t0\t0\t1\tNO_PINS\r\n'
text+=f'U\t{uid}\t\t\t\t\t\r\nG\t{uid}\t{enc(state)}\r\n'
path.write_bytes(text.encode())
output=guards.run_vbs(tmp_path,['QPReadSnapshot','QPReadUtf8'],'''
Dim QP_Fso,Records,A,B
Set QP_Fso=CreateObject("Scripting.FileSystemObject")
Set Records=QPReadSnapshot("'''+str(path)+'''")
Set A=New Annotation:A.UID="annotation1"
Set B=New Annotation:B.UID="annotation2"
WScript.Echo QPComponentIdentityKey(A)
WScript.Echo QPComponentIdentityKey(B)
WScript.Echo CStr(QP_ReferenceAliases.Exists("main:1|"))
''',prelude='''
Class Annotation
Public UID,Refdes
End Class
Function QPSheetIdentity()
QPSheetIdentity="MAIN:1"
End Function
''')
assert output.splitlines()==['annotation1','annotation2','False']
def test_semantic_capture_reports_each_completed_component_without_changing_baseline(tmp_path):
import test_accuracy_vbs_guards as guards
output=guards.run_vbs(tmp_path,['QPCurrentSemanticState','QPSortSnapshotLines'],'''
Dim QP_App,State
Set QP_App=New App
State=QPCurrentSemanticState()
WScript.Echo Replace(State,vbCrLf,";")
''',prelude='''
Const VDM_COMP=128
Const VD_ALL=0
Class App
Property Get ActiveView()
Set ActiveView=Me
End Property
Function Query(ByVal Kind,ByVal Scope)
Dim Rows:Set Rows=CreateObject("Scripting.Dictionary")
Rows.Add "C3",True:Rows.Add "C1",True:Rows.Add "C2",True
Set Query=Rows
End Function
End Class
Sub QPInvalidatePinIndex()
End Sub
Sub QPInvalidateComponentIndex()
End Sub
Function QPUntouchedFullState(ByVal Comp)
QPUntouchedFullState=Comp & vbCrLf
End Function
Sub QPProgress(ByVal Stage,ByVal Done,ByVal Total)
WScript.Echo Stage & ":" & Done & "/" & Total
End Sub
''')
assert output.splitlines()[:3]==['capture_semantic_state:1/3','capture_semantic_state:2/3','capture_semantic_state:3/3']
assert output.splitlines()[3]=='S\tfull_sheet_v1;C1;C2;C3;'
def test_semantic_component_progress_refreshes_business_watchdog():
from dx_productivity.trial_task_center.worker_monitor import ProgressWatch
watch=ProgressWatch('token','job',150,0)
for sequence,now in enumerate((100,200,300,400),1):
assert watch.accept(f'DX_PROGRESS\ttoken\tjob\t{sequence}\tcapture_semantic_state\t{sequence}\t104\t0\n'.encode(),now)
assert watch.action(now+100) is None
assert not watch.stop_requested
+9 -6
View File
@@ -10,8 +10,10 @@ from dx_productivity.trial_task_center.batch_resources import LIVE_RESOURCES, cl
from dx_productivity.trial_task_center.errors import TaskCenterError
def test_closed_desktop_rechecks_only_observe_until_external_reference_disappears(tmp_path):
def test_closed_desktop_rechecks_only_observe_until_external_reference_disappears(tmp_path, monkeypatch):
from dx_productivity.trial_task_center.isolated_desktop import PrivateDesktop
from dx_productivity.trial_task_center import isolated_desktop, batch_resources
import json
calls = []
class Api:
present = True
@@ -21,6 +23,7 @@ def test_closed_desktop_rechecks_only_observe_until_external_reference_disappear
if self.present:
raise OSError('external reference remains')
api = Api()
monkeypatch.setattr(isolated_desktop, '_Win32Api', lambda: api)
runtime = BatchCadRuntime(tmp_path, 'execution', desktop_factory=Desktop)
runtime._ensure_desktop()
desktop = PrivateDesktop(runtime.record['desktop'], enabled=True, api=api)
@@ -28,19 +31,19 @@ def test_closed_desktop_rechecks_only_observe_until_external_reference_disappear
runtime.desktop = desktop
runtime.result = 'saved result'
try:
with pytest.raises(TaskCenterError):
runtime.close()
original_error = runtime.record['error']
runtime.close()
assert runtime.record['state'] == 'isolated'
for _ in range(3):
assert cleanup_pending_resources(tmp_path)
assert not batch_resources.blocking_resources(tmp_path)
assert calls.count('close') == 1
api.present = False
assert cleanup_pending_resources(tmp_path) == ()
assert calls.count('close') == 1
assert all(call == 'close' or call == ('observe', desktop.name) for call in calls)
assert runtime.result == 'saved result'
assert runtime.record['error'] == original_error
assert runtime.record['cleanup']['desktop_confirmation']['released'] is True
final = json.loads(runtime.path.read_text(encoding='utf-8'))
assert final['state'] == 'released' and final['release_proof']['desktop_released'] is True
finally:
LIVE_RESOURCES.pop(str(runtime.path), None)
+162
View File
@@ -0,0 +1,162 @@
import json
from types import SimpleNamespace
import pytest
from dx_productivity.trial_task_center import batch_resources, batch_runtime, isolated_desktop
def runtime_with_residual(tmp_path, monkeypatch):
calls = []
class Desktop:
def __init__(self, name, **options):
self.name = name
self._processes = []
self.cleanup_record = None
def __enter__(self): return self
def close(self):
calls.append('close')
self.cleanup_record = {'desktop_handle_closed': True,
'owned_cleanup_complete': True,
'desktop_confirmation': {'released': False}, 'errors': ['foreign reference']}
raise OSError('foreign reference')
def release_desktop(self):
calls.append('observe')
raise OSError('foreign reference')
runtime = batch_runtime.BatchCadRuntime(tmp_path, 'execute_task_queue', desktop_factory=Desktop)
runtime.__enter__()
runtime._ensure_desktop()
runtime.result = {'completed': True}
return runtime, calls
def test_residual_desktop_is_warning_admits_fresh_batch_and_never_repeats_close(tmp_path, monkeypatch):
runtime, calls = runtime_with_residual(tmp_path, monkeypatch)
runtime.close()
assert runtime.record['state'] == 'isolated'
assert runtime.record['blocking'] is False
assert runtime.record['release_results']['desktop_released'] is False
assert batch_resources.blocking_resources(tmp_path) == ()
assert batch_resources.pending_resources(tmp_path)[0]['state'] == 'isolated'
with pytest.raises(batch_resources.BatchResourceCleanupError):
runtime._ensure_desktop()
runtime.close()
monkeypatch.setattr(isolated_desktop, '_Win32Api', lambda: SimpleNamespace(
confirm_desktop_released=lambda name: (_ for _ in ()).throw(OSError('foreign reference'))))
batch_resources.cleanup_pending_resources(tmp_path)
assert calls == ['close']
with batch_runtime.BatchCadRuntime(tmp_path, 'next') as admitted:
assert admitted.generation == 0
assert runtime.result == {'completed': True}
def test_isolation_proof_is_revalidated_after_restart_and_does_not_trust_label(tmp_path, monkeypatch):
runtime, _ = runtime_with_residual(tmp_path, monkeypatch)
identity = {'pid': 1234, 'creation_time': 'exact'}
runtime.record['writer_identities'] = [identity]
monkeypatch.setattr(batch_resources, '_process_gone', lambda item: True)
runtime.close()
proof = runtime.record['isolation_proof']
assert proof['processes'] == [identity]
assert proof['session_id'] == runtime.session_id and proof['generation'] == 1
assert batch_resources.blocking_resources(tmp_path) == ()
monkeypatch.setattr(batch_resources, '_process_gone', lambda item: False)
assert batch_resources.blocking_resources(tmp_path)
monkeypatch.setattr(isolated_desktop, '_Win32Api', lambda: SimpleNamespace(confirm_desktop_released=lambda _: None))
assert batch_resources.cleanup_pending_resources(tmp_path)
assert batch_resources.blocking_resources(tmp_path)
monkeypatch.setattr(batch_resources, '_process_gone', lambda item: True)
record = json.loads(runtime.path.read_text(encoding='utf-8'))
record['isolation_proof']['generation'] = 99
batch_resources.write_record(runtime.path, record)
assert batch_resources.blocking_resources(tmp_path)
def test_active_or_unknown_writer_cannot_be_isolated(tmp_path, monkeypatch):
runtime, _ = runtime_with_residual(tmp_path, monkeypatch)
runtime.record['writer_identities'] = [{'pid': 1234, 'creation_time': 'exact'}]
monkeypatch.setattr(batch_resources, '_process_gone', lambda item: False)
with pytest.raises(batch_resources.BatchResourceCleanupError): runtime.close()
assert batch_resources.blocking_resources(tmp_path)
runtime.record['writer_identities'][0]['creation_time'] = None
monkeypatch.setattr(batch_resources, '_process_gone', lambda item: True)
with pytest.raises(batch_resources.BatchResourceCleanupError): runtime.close(reason='manual_retry')
assert batch_resources.blocking_resources(tmp_path)
def test_isolation_log_never_claims_resources_released(tmp_path):
from dx_productivity.trial_task_center.batch_release_log import write_release_event
from dx_productivity.trial_task_center.logging_setup import configure_logging
context = configure_logging(tmp_path)
path = tmp_path / 'batch-resources' / ('a' * 32) / 'session.json'
record = {'session_id': 'a' * 32, 'generation': 1, 'state': 'isolated', 'kind': 'execution'}
assert write_release_event(tmp_path, path, record, 'isolated', context=context) == []
text = path.parent.joinpath('resource-release.log').read_text(encoding='utf-8')
assert 'isolated' in text
def test_passive_disappearance_confirmation_updates_both_release_state_and_result(tmp_path, monkeypatch):
runtime, calls = runtime_with_residual(tmp_path, monkeypatch)
runtime.close()
monkeypatch.setattr(isolated_desktop, '_Win32Api', lambda: SimpleNamespace(confirm_desktop_released=lambda _: None))
assert batch_resources.cleanup_pending_resources(tmp_path) == ()
final = json.loads(runtime.path.read_text(encoding='utf-8'))
assert final['release_results']['desktop_released'] is True
assert final['release_state']['desktop'] == 'released'
assert calls == ['close']
def test_supervisor_preserves_success_and_business_failure_with_nonfatal_desktop_warning(tmp_path, monkeypatch):
from dx_productivity.trial_task_center.batch_supervisor import _finish_response
from dx_productivity.trial_task_center.logging_setup import configure_logging
from dx_productivity.trial_task_center.supervisor_protocol import encode, encode_error
from dx_productivity.trial_task_center.errors import TaskCenterError
runtime, _ = runtime_with_residual(tmp_path, monkeypatch)
runtime.close()
warnings = batch_resources.pending_resources(tmp_path)
monkeypatch.setattr(batch_resources, 'cleanup_pending_resources', lambda *args, **kwargs: warnings)
journal = {'operation': 'probe', 'batch_id': runtime.session_id}
result = {'verified': True}
response = {'business_succeeded': True, 'status': 'ok', 'result': encode(result)}
assert _finish_response(tmp_path, journal, response, configure_logging(tmp_path)) == result
assert journal['state'] == 'finished'
assert journal['resource_recheck']['desktop_released'] is False
response = {'business_succeeded': False, 'status': 'error',
'error': encode_error(TaskCenterError('replacement_uncertain', 'requires verification'))}
with pytest.raises(TaskCenterError) as caught:
_finish_response(tmp_path, journal, response, configure_logging(tmp_path))
assert caught.value.code == 'replacement_uncertain'
assert caught.value.resources_released is False and caught.value.owned_writers_exited is True
def test_resource_status_displays_isolation_warning_without_blocking(qtbot):
from dx_productivity.trial_task_center.ui.resource_status import ResourceStatus
warning = {'state': 'isolated', 'blocking': False, 'desktop_released': False}
status = ResourceStatus(lambda: (warning,))
try:
status.refresh()
qtbot.waitUntil(lambda: not status.busy)
assert status.records == (warning,) and not status.blocked
status._received((warning, {'state': 'blocked'}))
assert status.blocked
finally:
status.close()
def test_failed_project_initiates_owned_cleanup_before_certifying_exit(tmp_path):
from test_batch_runtime import Desktop
from dx_productivity.trial_task_center.stop_control import StopControl
control = StopControl()
runtime = batch_runtime.BatchCadRuntime(tmp_path, 'execution', control, desktop_factory=Desktop)
runtime._ensure_desktop()
process = SimpleNamespace(returncode=None, poll=lambda: process.returncode)
def close(*args):
assert (tmp_path / 'failed' / 'uncertain.flag').exists()
assert control.cleanup_mode == 'mentor_release_required'
assert runtime.record['wait_reason'] == 'mentor_exit'
process.returncode = 0
runtime.cad = SimpleNamespace(cad=process, __exit__=close, record={})
assert runtime.release_failed_project(RuntimeError('source read failed'), tmp_path / 'failed')
assert runtime.desktop is None and runtime.cad is None
proof = json.loads((tmp_path / 'failed' / 'project-resource-release.json').read_text())
assert proof['writers_exited'] is True and proof['business_uncertainty_preserved'] is True
@@ -0,0 +1,104 @@
from pathlib import Path
import subprocess
import os
import sys
from threading import Event
import pytest
from dx_productivity.trial_task_center import batch_attempt
from dx_productivity.trial_task_center.stop_control import StopControl
from dx_productivity.trial_task_center.worker_monitor import WorkerMonitorError
@pytest.mark.parametrize('damage', [None, 'current_mutated', 'marker_missing', 'uncovered_prefix', 'source_unexplained', 'checkpoint'])
def test_prewrite_pause_certifies_only_current_job_and_hash_binds_committed_prefix(tmp_path, monkeypatch, damage):
from test_batch_runtime import command_for, worker_runtime, Cad
command, project = command_for(tmp_path)
root = Path(command[3])
text = (root / 'command.tsv').read_text()
(root / 'command.tsv').write_text(text + f'task\tnext\t{project}\told\tnew\n')
request = root / 'project-session.request'
request.write_text(f'{"a"*32}\t{project}\tjob')
def monitor(*args, **kwargs):
runtime.stop_event.set()
request.write_text(f'{"a"*32}\t{project}\tnext')
paths = {expected['job_id']: (path, expected) for path, expected in runtime._current_attempts}
path, expected = paths['job']
batch_attempt._publish(path, dict(expected, state='mutation_possible'))
if damage == 'current_mutated':
path, expected = paths['next']
batch_attempt._publish(path, dict(expected, state='mutation_possible'))
project.write_text('saved prefix')
if damage != 'marker_missing':
(root / ('project-pause-' + 'a'*32 + '.tsv')).write_text(f'{"a"*32}\t{project}\tnext\tprewrite_closed')
if damage != 'source_unexplained':
(root / ('project-commit-' + 'a'*32 + '.tsv')).write_text(
f'{"a"*32}\t{project}\tproject_verified\n' + ('foreign' if damage == 'uncovered_prefix' else 'job') + '\tsaved_ok\n')
for name in ('post-state-job.tsv', 'snapshot-job.tsv', 'snapshot-job.tsv.native.json', 'value-readback-job.tsv'):
(root / name).write_text('bound native prefix evidence')
if damage == 'checkpoint':
runtime.cad.checkpoint = lambda *args: (_ for _ in ()).throw(WorkerMonitorError('project still open'))
result = subprocess.CompletedProcess(command, 75, '', '')
result.prewrite_pause = {'session_token': 'a'*32, 'monitor_token': 'b'*32,
'job_id': 'next', 'project_path': str(project), 'sequence': 7, 'worker_tree_exited': True}
return result
runtime = worker_runtime(tmp_path, monkeypatch, monitor)
runtime.cad_factory = lambda *args, stop_control=None, **options: Cad(*args, **options)
runtime.stop_event = StopControl()
# Invalid certificates must fail promptly; no fake writer needs a live resolution wait.
monkeypatch.setattr(runtime, '_retain_uncertain', lambda *args: None)
with runtime:
if damage:
with pytest.raises(WorkerMonitorError): runtime.run_worker(command, 5)
else:
result = runtime.run_worker(command, 5)
proof = result.prewrite_pause
assert proof['phase'] == 'before_native_dispatch' and proof['job_id'] == 'next'
assert len(proof['batch_attempts']) == 1
assert proof['batch_attempts'][0]['expected']['job_id'] == 'next'
assert proof['committed_prefix'] == {'job': 'saved_ok'}
assert proof['native_dispatch_not_started'] and proof['project_closed']
assert proof['files']
@pytest.mark.skipif(os.name != 'nt', reason='Actual owned Windows protocol actor')
@pytest.mark.parametrize('damage', [None, 'token', 'session', 'job', 'sequence', 'marker', 'no_abort'])
def test_monitor_prewrite_pause_correlates_current_job_after_prior_unsafe_prefix(tmp_path, damage):
from dx_productivity.trial_task_center.worker_monitor import run_monitored_process
from dx_productivity.trial_task_center.isolated_desktop import _Win32Api
project = tmp_path / 'board.prj'
project.write_text('fixture')
(tmp_path / 'project-session.request').write_text(f'{"a"*32}\t{project}\tjob')
script = tmp_path / 'actor.py'
script.write_text('''import pathlib,sys,time
root=pathlib.Path(sys.argv[1]);job=sys.argv[4];token=sys.argv[5]
damage=''' + repr(damage) + '''
while not (root/f'monitor-{token}.ready').exists():time.sleep(.002)
if damage!='no_abort':
while not (root/f'monitor-{token}.stop').exists():time.sleep(.002)
request=root/'project-session.request'
fields=request.read_text().split('\\t');fields[2]='next';request.write_text('\\t'.join(fields))
print(f'DX_PROGRESS\\t{token}\\t{job}\\t1\\tnative_replace\\t0\\t1\\t0\\tjob',flush=True)
print(f'DX_PROGRESS\\t{token}\\t{job}\\t2\\tverify_saved_project\\t1\\t1\\t0\\tjob',flush=True)
if damage!='marker':(root/('project-pause-'+fields[0]+'.tsv')).write_text('\\t'.join([fields[0],fields[1],'next','prewrite_closed']))
print('\\t'.join(['DX_PREWRITE_PAUSED','bad' if damage=='token' else token,job,
'c'*32 if damage=='session' else fields[0],'bad' if damage=='job' else 'next',
'99' if damage=='sequence' else '2']),flush=True)
sys.exit(75)
''', encoding='utf-8')
stop, api = Event(), _Win32Api()
if damage != 'no_abort': stop.set()
command = (sys.executable, '-u', str(script), str(tmp_path), sys.executable, '', 'job')
try:
if damage:
with pytest.raises(WorkerMonitorError):
run_monitored_process(command, 5, abort_event=stop,
process_factory=lambda launch, **options: api.create_process(launch, None, tmp_path, None, **options))
else:
result = run_monitored_process(command, 5, abort_event=stop,
process_factory=lambda launch, **options: api.create_process(launch, None, tmp_path, None, **options))
assert result.prewrite_pause['job_id'] == 'next'
assert result.prewrite_pause['worker_tree_exited'] is True
finally:
api.close_pending()
+3 -3
View File
@@ -54,7 +54,7 @@ def test_restart_with_incomplete_handoff_never_replays_worker(resume_case,damage
assert context.interrupted
assert context.has_evidence
outcome=execute_task_queue(restarted,(task.task_id,),stop_event=Event())
assert outcome.paused and outcome.entries[0].state=='blocked'
assert not outcome.paused and outcome.entries[0].state=='blocked'
assert len(calls)==2
@@ -84,7 +84,7 @@ def test_receipts_block_rollback_to_an_empty_status_even_without_sidecars(resume
restarted=reopen()
assert restarted.task_context(task.task_id).interrupted
result=execute_task_queue(restarted,(task.task_id,),stop_event=Event())
assert result.paused and len(calls)==2
assert not result.paused and result.entries[0].state=='blocked' and len(calls)==2
def test_missing_entire_handoff_cannot_erase_database_receipts(resume_case):
@@ -93,4 +93,4 @@ def test_missing_entire_handoff_cannot_erase_database_receipts(resume_case):
restarted=reopen()
assert restarted.task_context(task.task_id).interrupted
result=execute_task_queue(restarted,(task.task_id,),stop_event=Event())
assert result.paused and len(calls)==2
assert not result.paused and result.entries[0].state=='blocked' and len(calls)==2
+2 -1
View File
@@ -89,7 +89,8 @@ def test_timeout_without_complete_proof_stays_uncertain(component_case, unsafe):
raise error
preparer._process_runner = worker
result = execute_task_queue(service, (task_id,), stop_event=stop)
assert result.paused
assert result.paused is (unsafe == 'cancelled')
assert result.entries[0].state == 'blocked'
assert calls == []
assert service.reserve_problem_retry(task_id) == ()
assert preparer.has_interrupted(task_id)
+7 -7
View File
@@ -70,11 +70,11 @@ def test_changed_confirmed_plan_is_rejected_before_any_executor(tmp_path):
assert not service.calls
def test_preflight_named_error_after_dispatch_needs_receipt_before_continuing(tmp_path):
def test_dispatched_error_isolates_the_project_after_its_owned_resources_exit(tmp_path):
service = Service(tmp_path, {'A': TaskCenterError('quick_run_preflight', 'target absent')})
result = execute_task_queue(service, ('A', 'B'), stop_event=Event())
assert result.paused and result.entries[0].state == 'blocked'
assert service.calls == [('start', 'A')]
assert not result.paused and result.entries[0].state == 'blocked'
assert service.calls == [('start', 'A'), ('start', 'B'), ('end', 'B')]
def test_actual_read_phase_failure_allows_next_project(tmp_path):
@@ -93,12 +93,12 @@ def test_actual_read_phase_failure_allows_next_project(tmp_path):
assert service.calls == [('start', 'B'), ('end', 'B')]
def test_uncertain_result_blocks_next_task(tmp_path):
def test_uncertain_result_is_quarantined_without_replaying_before_next_independent_task(tmp_path):
service = Service(tmp_path, {'A': TaskCenterError('quick_run_uncertain', 'unknown')})
result = execute_task_queue(service, ('A', 'B'), stop_event=Event())
assert result.paused
assert not result.paused
assert result.entries[0].state == 'blocked'
assert service.calls == [('start', 'A')]
assert service.calls == [('start', 'A'), ('start', 'B'), ('end', 'B')]
def test_pause_finishes_current_item_and_does_not_start_next_task(tmp_path):
@@ -122,7 +122,7 @@ def test_pause_between_problem_tasks_preserves_resumable_batch(tmp_path, monkeyp
assert result.paused and result.current_task_id in {'A', 'B'}
assert [entry.state for entry in result.entries] == ['failed', 'completed']
import json
checkpoint = json.loads((tmp_path / 'task-queues' / (result.queue_id + '.json')).read_text())
checkpoint = json.loads((tmp_path / 'task-queues' / (result.queue_id + '.json')).read_text(encoding='utf-8'))
assert checkpoint['phase'] == 'problems'
+114
View File
@@ -0,0 +1,114 @@
"""Resource recheck failures must publish a terminal result after writers exit."""
import json
import os
from pathlib import Path
import subprocess
import sys
import textwrap
from types import SimpleNamespace
from uuid import uuid4
import pytest
from dx_productivity.trial_task_center import batch_worker, isolated_cad, stop_control, worker_monitor
from dx_productivity.trial_task_center.batch_resources import BatchResourceCleanupError
from dx_productivity.trial_task_center.supervisor_protocol import encode
@pytest.mark.parametrize('owner', ['exited', 'runtime', 'cad', 'worker'])
def test_resource_error_is_returnable_only_after_all_write_owners_exit(tmp_path, monkeypatch, owner):
failure = BatchResourceCleanupError('desktop object still exists')
def close(**options):
raise failure
runtime = SimpleNamespace(read_only=False, root=tmp_path, path=tmp_path / 'session.json',
record={'state': 'blocked'}, desktop=object(), cad=None,
owned_processes_exited=lambda: owner != 'runtime', close=close)
monkeypatch.setattr(isolated_cad, 'RETAINED_CAD_SESSIONS', {
'fixture': SimpleNamespace(root=tmp_path, _owned_processes=lambda: [
SimpleNamespace(tree_exited=lambda: owner != 'cad')])})
monkeypatch.setattr(worker_monitor, 'RETAINED_WORKERS', {
'fixture': {'root': tmp_path, 'process': object(), 'owned': object()}})
monkeypatch.setattr(worker_monitor, '_worker_tree_exited', lambda *args: owner != 'worker')
monkeypatch.setattr(worker_monitor, 'cleanup_retained_workers', lambda *args, **kwargs: None)
# Simulate a failed release attempt; it must not count as proof of exit.
monkeypatch.setattr(stop_control, 'wait_for_resolution', lambda *args, **kwargs: None)
if owner == 'exited':
assert batch_worker._wait_for_owned_writes(runtime, stop_control.StopControl()) is failure
else:
with pytest.raises(BatchResourceCleanupError) as caught:
batch_worker._wait_for_owned_writes(runtime, stop_control.StopControl())
assert caught.value is failure
@pytest.mark.parametrize('business_failed', [False, True])
@pytest.mark.parametrize('initial_cleanup_failed', [False, True])
def test_real_worker_exits_with_durable_result_when_desktop_recheck_fails(tmp_path, business_failed,
initial_cleanup_failed):
batch_id, attempt_id = uuid4().hex, uuid4().hex
directory = tmp_path / 'batch-resources' / batch_id
directory.mkdir(parents=True)
request = directory / 'request.json'
request.write_text(json.dumps({'version': 1, 'batch_id': batch_id, 'attempt_id': attempt_id,
'root': str(tmp_path), 'database': str(tmp_path / 'tasks.db'), 'operation': 'execute_task_queue',
'arguments': encode({}), 'task_ids': []}), encoding='utf-8')
launcher = textwrap.dedent('''
import sys
from pathlib import Path
from types import SimpleNamespace
from dx_productivity.trial_task_center import batch_runtime, batch_worker, read_recovery, task_queue
from dx_productivity.trial_task_center.batch_resources import BatchResourceCleanupError
from dx_productivity.trial_task_center.store import TaskStore
from dx_productivity.trial_task_center.errors import TaskCenterError
batch_worker._admit_request = lambda *args: {}
read_recovery.check_owner_loss = lambda *args, **kwargs: False
class FailingDesktop(batch_runtime.BatchCadRuntime):
def __enter__(self):
self.desktop = SimpleNamespace(_processes=[], cleanup_record={
'desktop_handle_closed': True, 'desktop_confirmation': {'released': False}})
self.record.update(state='blocked', error='desktop object still exists')
return self
def __exit__(self, *args):
if sys.argv[3] == 'True':
raise BatchResourceCleanupError('initial desktop release failed')
def close(self, **kwargs):
raise BatchResourceCleanupError('desktop recheck still failed')
batch_runtime.BatchCadRuntime = FailingDesktop
def execute(*args, **kwargs):
if sys.argv[2] == 'True':
raise TaskCenterError('fixture', 'business failed')
return task_queue.TaskQueueResult('fixture', (
task_queue.TaskQueueEntry('saved', 'completed'),
task_queue.TaskQueueEntry('untouched', 'pending')), paused=True)
task_queue.execute_task_queue = execute
close_store = TaskStore.close
def close(self):
close_store(self)
(Path(sys.argv[1]).parent / 'store-closed').touch()
TaskStore.close = close
raise SystemExit(batch_worker.run(Path(sys.argv[1])))
''')
result = subprocess.run([sys.executable, '-X', 'utf8', '-c', launcher, str(request),
str(business_failed), str(initial_cleanup_failed)],
capture_output=True, text=True, encoding='utf-8', timeout=20,
env=dict(os.environ, PYTHONPATH=str(Path(batch_worker.__file__).resolve().parents[2])))
assert result.returncode == 0, result.stderr
final = json.loads((directory / 'result.json').read_text(encoding='utf-8'))
body = final['payload']
assert final['batch_id'] == batch_id and final['attempt_id'] == attempt_id
assert body['status'] == 'error' and body['business_succeeded'] is (not business_failed)
assert (directory / 'store-closed').is_file()
assert json.loads(result.stdout.splitlines()[-1]) == final
assert 'desktop recheck still failed' in (directory / 'supervisor.log').read_text(encoding='utf-8')
expected_error = ('initial desktop release failed' if initial_cleanup_failed else
'business failed' if business_failed else 'desktop recheck still failed')
assert body['error']['message'] == expected_error
if business_failed:
assert body['operation_error']['message'] == 'business failed'
else:
saved = json.loads((directory / 'business-result.json').read_text(encoding='utf-8'))
assert body['result'] == saved['payload']['result']
from dx_productivity.trial_task_center.supervisor_result import validated_outcome
queue = validated_outcome('execute_task_queue', body)[0]
assert queue.paused is True
assert [(entry.task_id, entry.state) for entry in queue.entries] == [
('saved', 'completed'), ('untouched', 'pending')]
+16 -5
View File
@@ -27,6 +27,16 @@ from dx_productivity.trial_task_center.task_queue import execute_task_queue
from dx_productivity.trial_task_center.ui.messages import _format_task_log_lines
class SimulatedReleasedRuntime:
"""The injected executor starts no owned process; admission proof is explicit."""
def __init__(self, root):
self.root, self.stop_event, self.cad, self.desktop = root, None, None, None
def bind_logs(self, *args):
pass
def release_failed_project(self, error, evidence_root):
return True
def run_scenario(output: Path, *, projects=3, sheets=2, rules=2, behavior='success') -> dict:
output = Path(output).resolve()
output.mkdir(parents=True, exist_ok=False)
@@ -126,17 +136,18 @@ def run_scenario(output: Path, *, projects=3, sheets=2, rules=2, behavior='succe
preview = measured('import_bom', lambda: service.preview_bom_plan(task.task_id, bom))
measured('save_plan', lambda: service.replace_target_plan(task.task_id, preview))
ids = tuple(task.task_id for task in tasks)
result = measured('queue_wall', lambda: execute_task_queue(service, ids, stop_event=stop))
simulated_runtime = SimulatedReleasedRuntime(service._logging.root)
result = measured('queue_wall', lambda: execute_task_queue(service, ids, stop_event=stop, runtime=simulated_runtime))
if behavior == 'pause_resume':
assert result.paused and len(calls) == 2
stop.clear()
result = measured('queue_resume_wall', lambda: execute_task_queue(service, ids, stop_event=stop, queue_id=result.queue_id))
result = measured('queue_resume_wall', lambda: execute_task_queue(service, ids, stop_event=stop, queue_id=result.queue_id, runtime=simulated_runtime))
expected_steps = projects * sheets * rules * 2
expected_calls = 1 if behavior == 'uncertain_stop' else expected_steps
expected_calls = 1 + (projects - 1) * sheets * rules * 2 if behavior == 'uncertain_stop' else expected_steps
assert len(calls) == expected_calls
if behavior == 'uncertain_stop':
assert result.paused and result.entries[0].state == 'blocked'
assert all(entry.state == 'pending' for entry in result.entries[1:])
assert not result.paused and result.entries[0].state == 'blocked'
assert all(entry.state == 'completed' for entry in result.entries[1:])
else:
assert not result.paused
assert [entry.state for entry in result.entries] == ['failed' if behavior == 'failure_continue' and i == 0 else 'completed' for i in range(projects)]
+72 -18
View File
@@ -14,11 +14,14 @@ from uuid import uuid4
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'src'))
from dx_productivity.trial_task_center.batch_resources import (
BatchResourceCleanupError, cleanup_pending_resources, pending_resources,
BatchResourceCleanupError, cleanup_pending_resources, pending_resources, blocking_resources,
process_identity, write_record,
)
from dx_productivity.trial_task_center.batch_runtime import BatchCadRuntime
from dx_productivity.trial_task_center.batch_worker import _wait_for_owned_writes
from dx_productivity.trial_task_center.cad_identity import current_desktop_name
from dx_productivity.trial_task_center.isolated_desktop import _Win32Api
from dx_productivity.trial_task_center.stop_control import StopControl
HOLDER = '''import json,os,sys,win32service
@@ -65,30 +68,81 @@ def probe(output):
handshake = json.loads(holder.stdout.readline())
assert handshake['pid'] == holder.pid and handshake['state'] == 'holding'
report['holder_fixture'] = handshake
try:
runtime.close()
raise AssertionError('cleanup incorrectly succeeded with external holder')
except BatchResourceCleanupError as failure:
assert failure.completed_result == original_result
report['events'].append({'phase': 'initial_cleanup', 'blocked': True,
'error': str(failure), 'holder_still_running': holder.poll() is None})
runtime.close()
report['events'].append({'phase': 'initial_cleanup', 'blocked': False,
'state': runtime.record['state'], 'holder_still_running': holder.poll() is None})
assert holder.poll() is None
blocked = pending_resources(root)
assert len(blocked) == 1 and blocked[0]['state'] == 'blocked'
assert len(blocked) == 1 and blocked[0]['state'] == 'isolated' and not blocking_resources(root)
assert blocked[0]['blocking'] is False and blocked[0]['release_results']['desktop_released'] is False
assert blocked[0]['cleanup']['desktop_confirmation']['remaining_windows'] == []
assert blocked[0]['cleanup']['desktop_handle_closed']
report['blocked_resource_record'] = blocked[0]
candidate = BatchCadRuntime(root, 'must_be_blocked', cad_factory=forbidden_cad)
report['isolated_resource_record'] = blocked[0]
# A synthetic persistent claim with an actual live owned helper proves
# that neither the isolated label nor a forged exit flag opens the gate.
active = subprocess.Popen((sys.executable, '-c', 'import sys;sys.stdin.readline()'),
stdin=subprocess.PIPE, creationflags=subprocess.CREATE_NO_WINDOW)
try:
candidate.__enter__()
raise AssertionError('new batch bypassed pending resource guard')
except BatchResourceCleanupError as failure:
report['events'].append({'phase': 'new_batch', 'blocked': True, 'error': str(failure)})
assert candidate.generation == 0 and candidate.desktop is None
active_identity = process_identity(active)
active_root = output / 'active-writer-data'
active_session = uuid4().hex
active_path = active_root / 'batch-resources' / active_session / 'session.json'
active_record = dict(blocked[0], session_id=active_session,
writer_identities=[active_identity], processes=[active_identity])
active_record.pop('record_path', None)
active_record['isolation_proof'] = dict(active_record['isolation_proof'],
session_id=active_session, processes=[active_identity])
write_record(active_path, active_record)
assert blocking_resources(active_root)
assert cleanup_pending_resources(active_root) and blocking_resources(active_root)
assert active.poll() is None
report['active_writer_gate'] = {'process': active_identity, 'blocked': True,
'record_fixture': 'synthetic isolated claim; real owned helper remains alive'}
finally:
active.stdin.write(b'release\n')
active.stdin.flush()
active.wait(timeout=5)
active.stdin.close()
assert not blocking_resources(active_root) # Exact recorded process has now exited.
candidate = BatchCadRuntime(root, 'fresh_batch', cad_factory=forbidden_cad)
with candidate:
fresh = candidate._ensure_desktop()
assert fresh.name != desktop.name
report['fresh_desktop'] = fresh.name
report['events'].append({'phase': 'new_batch', 'blocked': False, 'fresh_desktop_created': True})
restart_code = '''import json,sys
from pathlib import Path
sys.path.insert(0,sys.argv[2])
from dx_productivity.trial_task_center.batch_resources import cleanup_pending_resources,blocking_resources
from dx_productivity.trial_task_center.batch_runtime import BatchCadRuntime
root=Path(sys.argv[1])
rows=cleanup_pending_resources(root)
assert rows and all(row['state']=='isolated' and row['blocking'] is False for row in rows)
assert not blocking_resources(root)
with BatchCadRuntime(root,'restart_admission') as runtime:
desktop=runtime._ensure_desktop()
assert desktop.name != rows[0]['desktop']
print(json.dumps({'restart_isolation_validated':True,'fresh_desktop':desktop.name}))
'''
restarted = subprocess.run((sys.executable, '-c', restart_code, str(root),
str(Path(__file__).resolve().parents[1] / 'src')), capture_output=True, text=True,
encoding='utf-8', timeout=30, creationflags=subprocess.CREATE_NO_WINDOW)
assert restarted.returncode == 0, restarted.stderr
report['restart'] = json.loads(restarted.stdout.strip())
cleanup_error = _wait_for_owned_writes(runtime, StopControl())
assert cleanup_error is None
assert runtime.owned_processes_exited() and holder.poll() is None
assert runtime.record['state'] == 'isolated' and result_path.read_bytes() == original_bytes
report['events'].append({'phase': 'worker_exit_resource_recheck',
'error_returned': False, 'owned_writers_exited': True,
'holder_still_running': True, 'resource_state': runtime.record['state']})
pending = cleanup_pending_resources(root)
assert pending and pending[0]['state'] == 'blocked' and holder.poll() is None
assert pending and pending[0]['state'] == 'isolated' and holder.poll() is None
assert not blocking_resources(root)
close_history = runtime.record['cleanup']['desktop_handle_history']
assert len([event for event in close_history if event['operation'] == 'close']) == 1
report['events'].append({'phase': 'resource_only_retry_while_held',
'blocked': True, 'holder_still_running': True})
'blocked': False, 'holder_still_running': True, 'owned_desktop_handle_closed_once': True})
assert result_path.read_bytes() == original_bytes
holder.stdin.write('release\n')
holder.stdin.flush()