Merge branch 'codex/review-native-backend' into codex/review-ui-integration

This commit is contained in:
阿白叔 committed 2026-11-09 14:02:14 +08:00
commit 0fca6abb4e
13 files changed
+152 -58

No files matched your search

@@ -9,3 +9,12 @@ Dependency rationale: library release imports native engineering identity and re
Rollback: before dependents are merged, revert this commit. After merging dependents, revert in reverse order or use the previous candidate package after closing the current app. Code rollback does not undo a native transaction; use a consistent manual backup or explicitly selected native history on an isolated copy.
Validation: full Python suite 1,376 passed (one existing pywinauto COM-threading warning); compileall and git diff --check passed. Native field recovery, Windows 10 and cross-machine shared-library acceptance remain pending.
Review comment 344 follow-up (2026-11-09):
- Production status scans continue to query fresh bridge results. The low-level session now also defaults to fresh scans; explicit checkpoint reuse is documented as a controlled read-only option with no production resume promise.
- `library-changes.json` declares `rule_version`, the selected `scope`, excluded directory categories and suffixes, positional runtime rules, runtime filenames/prefixes, and recognized coordination checkpoints. Names, suffixes, prefixes and native database locations come from the same constants used by semantic classification. These are differences in participating engineering files; excluded content is not proven unchanged, and unrelated CBB content is not checked.
- Removed the tests-only `pcb_bridge.execution_outcome` helper. Production release safety continues to come from observed release state and remaining clients.
- Unlocked primary snapshots hash the copy stream, then independently hash the final source and destination. Source reads decrease from three to two. Rejected the suggestion to remove/threshold the final source hash: size and mtime cannot detect same-size, same-timestamp source edits. Locked LMC/icdb streams and owned lock snapshots retain their existing behavior; no source-rehash skip flag is added.
Validation for this follow-up: 287 targeted cases passed through the isolated mutex/PYTHONPATH wrapper (`test_batch_status`, `test_batch_status_faults`, `test_review_backend_followup`, `test_scoped_manifest`, `test_pcb_bridge`, `test_management_retry_blockers`, `test_backup_recovery`, `test_library_lock`, `test_management`). AST parsing, `git diff --check`, and original newline-style checks passed. No real Mentor, COM automation, shared library or process control was exercised; no full-suite run was requested.
@@ -1,5 +1,7 @@
# CBB Batch Status Bridge Implementation Plan
Current behavior (2026-11-09): production status scans always query the bridge fresh and overwrite checkpoint evidence. `BatchStatusSession.scan` defaults to `resume=False`; `resume=True` remains an explicit low-level option for controlled read-only scans, not an exposed production resume feature. Its lightweight fingerprint cannot detect every native payload/database write, so saved READY/PENDING rows must not authorize production actions.
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Read 180 to 3000 CBB Verify states through one resilient x86 Mentor bridge session with checkpoint recovery and authoritative `READY/PENDING` results.
@@ -1,5 +1,7 @@
# CBB 批量状态桥设计
Current behavior (2026-11-09): production status scans always query the bridge fresh and overwrite checkpoint evidence. `BatchStatusSession.scan` defaults to `resume=False`; `resume=True` remains an explicit low-level option for controlled read-only scans, not an exposed production resume feature. Its lightweight fingerprint cannot detect every native payload/database write, so saved READY/PENDING rows must not authorize production actions.
## 目标
支持对 180 到 3000 个 CBB 进行 Mentor authoritative Verify 状态读取,要求单条失败隔离、过程可恢复、结果可审计,并避免现有“一条 CBB 启动一个桥进程”的启动开销。
+10 -3
View File
@@ -107,7 +107,7 @@ def fingerprint_library(lmc: Path) -> LmcFingerprint:
# Metadata alone is not a safe checkpoint key: on network shares and on
# file systems with coarse timestamp resolution, two different LMC
# contents can have the same size and mtime. Include the actual file
# digest so a changed central library can never reuse an old status row.
# digest to invalidate changed LMC bytes; payload edits remain outside this key.
content_sha256 = _sha256_file(path)
combined = hashlib.sha256()
combined.update(str(path).casefold().encode('utf-8', errors='surrogatepass'))
@@ -186,7 +186,12 @@ def _now() -> str:
class BatchStatusSession:
"""Supervised serial bridge session with checkpoint recovery."""
"""Supervised serial bridge session; fresh status queries by default.
Checkpoints record scan evidence. Explicit ``resume=True`` is a low-level
opt-in for controlled read-only scans, not a production freshness proof:
the lightweight fingerprint does not cover native payload/database edits.
"""
def __init__(self, bridge: StatusBridgeClient, *, timeout_seconds: float = 10.0,
rotate_every: int = 500) -> None:
@@ -196,7 +201,7 @@ class BatchStatusSession:
self.restart_count = 0
def scan(self, names: Iterable[str], lmc_file: Path, checkpoint_path: Path,
*, block_type: str = "RULP", resume: bool = True) -> list[BatchStatusRecord]:
*, block_type: str = "RULP", resume: bool = False) -> list[BatchStatusRecord]:
fingerprint = fingerprint_library(Path(lmc_file))
if resume:
checkpoint = BatchCheckpoint.load(checkpoint_path)
@@ -291,6 +296,8 @@ def authoritative_status_map(names: Iterable[str], lmc_file: Path, checkpoint_pa
timeout_seconds: float = 10.0) -> dict[str, str]:
"""Return only bridge-backed statuses for an optional UI integration.
Production always queries fresh statuses, including after native writes;
saved READY/PENDING rows are never reused here.
A missing bridge is represented as ``UNAVAILABLE``/``UNKNOWN`` by callers;
it is never promoted to ``READY``.
"""
+2 -1
View File
@@ -8,7 +8,7 @@ from uuid import uuid4
from .library_lock import LibraryLock, LibraryBusy
from .scoped_manifest import (ManifestScope, build_manifest, build_exclusive_library_manifest,
copy_verified, hash_file, scan_tree)
comparison_rules, copy_verified, hash_file, scan_tree)
@dataclass(frozen=True)
@@ -212,6 +212,7 @@ def run_management(request, *, adapter=None, verifier=None, idle_check=None, pro
nonlocal change_report
if mutated and original is not None and change_report is None:
change_report = {'status': 'UNCONFIRMED', 'scope': original['scope'],
**comparison_rules(scope),
'unrelated_cbb_content_checked': False,
'detail': '原生会话尚未释放,未读取写后内容'}
if safe:
-15
View File
@@ -160,18 +160,3 @@ def wait_for_target_document(job, app_provider, *, process_alive, emit=lambda **
last_observation = observation
sleep(.25)
raise PcbBridgeError('PCB_EXITED_BEFORE_COM', 'PCB 在 COM 文档就绪前退出')
def execution_outcome(*, save_complete: bool, pcb_exited: bool,
fatal_error: str = '') -> dict[str, object]:
if fatal_error:
code, _, detail = str(fatal_error).partition(':')
return {'status': 'FAIL', 'error_code': code.strip() or 'FATAL_DIALOG',
'stderr': detail.strip() or str(fatal_error), 'safe_to_continue': True}
if not save_complete:
return {'status': 'FAIL', 'error_code': 'SAVE_NOT_CONFIRMED',
'stderr': '保存/关闭未完成', 'safe_to_continue': True}
if not pcb_exited:
return {'status': 'FAIL', 'error_code': 'PCB_EXIT_NOT_CONFIRMED',
'stderr': '保存后未确认 PCB 退出', 'safe_to_continue': True}
return {'status': 'PASS', 'stdout': 'Project Integration 已同步;保存、关闭和 PCB 退出均已确认。'}
+59 -11
View File
@@ -15,6 +15,15 @@ import stat
MANIFEST_VERSION = 1
MANIFEST_RULE_VERSION = 1
_ROOT_RUNTIME_PREFIX = 'work.before-restore-'
_ROOT_RUNTIME_PATH = ('projectbackup', 'backups')
_NATIVE_RUNTIME_DIRECTORIES = ('log', 'dbcheck', 'dbrestore')
_NATIVE_SERVER_DIRECTORY = 'cdbsvr'
_PROJECT_PCB_DIRECTORY = 'pcb'
_PROJECT_DATABASE_PREFIXES = (('database',), (_PROJECT_PCB_DIRECTORY, 'database'))
_NONPRIMARY_FILENAMES = ('saddress.adr',)
_NONPRIMARY_FILENAME_PREFIXES = ('~$',)
_NONPRIMARY_DIRECTORIES = frozenset({
'log', 'logs', 'temp', 'tmp', '__pycache__', 'backup', 'backups',
'output', 'outputs', 'report', 'reports', 'gerber', 'gerbers', 'odb++',
@@ -29,22 +38,21 @@ def _root_runtime(directories):
"""Known runtime/history children of a library or project workspace root."""
return bool(directories and (
directories[0] in _NONPRIMARY_DIRECTORIES
or directories[0].startswith('work.before-restore-')
or directories[:2] == ('projectbackup', 'backups')))
or directories[0].startswith(_ROOT_RUNTIME_PREFIX)
or directories[:len(_ROOT_RUNTIME_PATH)] == _ROOT_RUNTIME_PATH))
def _native_runtime(directories, database_prefix):
runtime_prefix = database_prefix + ('cdbsvr',)
runtime_prefix = database_prefix + (_NATIVE_SERVER_DIRECTORY,)
return (directories[:len(runtime_prefix)] == runtime_prefix
and len(directories) > len(runtime_prefix)
and directories[len(runtime_prefix)] in ('log', 'dbcheck', 'dbrestore'))
and directories[len(runtime_prefix)] in _NATIVE_RUNTIME_DIRECTORIES)
def _project_runtime(directories):
return (_root_runtime(directories)
or (directories[:1] == ('pcb',) and _root_runtime(directories[1:]))
or _native_runtime(directories, ('database',))
or _native_runtime(directories, ('pcb', 'database')))
or (directories[:1] == (_PROJECT_PCB_DIRECTORY,) and _root_runtime(directories[1:]))
or any(_native_runtime(directories, prefix) for prefix in _PROJECT_DATABASE_PREFIXES))
@dataclass(frozen=True)
@@ -85,7 +93,8 @@ class ManifestScope:
"""
directories = parts if directory else parts[:-1]
if not directory and (Path(parts[-1]).suffix in _NONPRIMARY_SUFFIXES
or parts[-1] == 'saddress.adr' or parts[-1].startswith('~$')):
or parts[-1] in _NONPRIMARY_FILENAMES
or parts[-1].startswith(_NONPRIMARY_FILENAME_PREFIXES)):
return True
if self.kind == 'project':
return _project_runtime(directories)
@@ -123,6 +132,36 @@ class ManifestScope:
return 'shared_dependency'
def comparison_rules(scope):
"""Describe semantic exclusions using the classifier's rule constants."""
project_positions = ['project root or its PCB child']
project_native = ' or '.join('/'.join(prefix + (_NATIVE_SERVER_DIRECTORY,))
for prefix in _PROJECT_DATABASE_PREFIXES)
native_positions = [project_native + ' under project root']
if scope.kind == 'library':
project_positions = ['library root', 'selected CBB Work/Cert root or its PCB child']
native_positions = [
project_native + ' under selected CBB Work/Cert',
'SymbolLibs/<partition>/' + _NATIVE_SERVER_DIRECTORY,
'<shared dependency>/' + _NATIVE_SERVER_DIRECTORY,
]
return {
'rule_version': MANIFEST_RULE_VERSION,
'excluded_categories': sorted(_NONPRIMARY_DIRECTORIES),
'excluded_suffixes': sorted(_NONPRIMARY_SUFFIXES),
'runtime_exclusions': {
'directory_name_positions': project_positions,
'directory_prefixes': [_ROOT_RUNTIME_PREFIX],
'relative_directories': ['/'.join(_ROOT_RUNTIME_PATH)],
'native_directory_names': sorted(_NATIVE_RUNTIME_DIRECTORIES),
'native_directory_positions': native_positions,
'file_names': list(_NONPRIMARY_FILENAMES),
'file_name_prefixes': list(_NONPRIMARY_FILENAME_PREFIXES),
'coordination_checkpoints': 'recognized CBB coordination checkpoints',
},
}
def hash_file(path):
digest = hashlib.sha256()
with Path(path).open('rb') as stream:
@@ -271,7 +310,10 @@ def copy_verified(source, destination, *, scope=None, progress=None, hash_functi
Native icdb.dat/LMC files are held with exclusive read handles for the copy.
Their expected SHA256 comes from the source read stream, independently of
hashing destination bytes. Failures retain the partial destination evidence.
hashing destination bytes. Unlocked primary files are also hashed while
streaming the copy, then rehashed at source to detect even same-size/mtime
edits. Every primary destination is independently hashed. Failures retain
the partial destination evidence.
Progress receives one dict with stage, hashed_files/bytes and copied_files.
"""
import win32con
@@ -329,9 +371,15 @@ def copy_verified(source, destination, *, scope=None, progress=None, hash_functi
shutil.copystat(path, copied)
if stream_hash is not None:
digest = stream_hash.hexdigest()
elif label is not None:
stream_hash = hashlib.sha256()
with path.open('rb') as incoming, copied.open('wb') as outgoing:
for data in iter(lambda: incoming.read(1024 * 1024), b''):
stream_hash.update(data)
outgoing.write(data)
shutil.copystat(path, copied)
digest = stream_hash.hexdigest()
else:
if label is not None:
digest = hash_function(path)
shutil.copy2(path, copied)
before, after = entries[path], path.stat()
if (before.st_size, before.st_mtime_ns) != (after.st_size, after.st_mtime_ns):
+21 -2
View File
@@ -84,11 +84,30 @@ def test_scan_reuses_checkpoint_and_queries_only_missing(tmp_path):
encoding="utf-8",
)
bridge = FakeBridge()
records = BatchStatusSession(bridge).scan(["A", "B"], lmc, checkpoint)
records = BatchStatusSession(bridge).scan(["A", "B"], lmc, checkpoint, resume=True)
assert bridge.requests == ["B"]
assert {record.block_name for record in records} == {"A", "B"}
def test_default_scan_refreshes_cached_ready_after_payload_change(tmp_path):
lmc = tmp_path/'library.lmc'; lmc.write_bytes(b'library')
checkpoint = tmp_path/'scan.jsonl'
BatchStatusSession(FakeBridge()).scan(['A'], lmc, checkpoint)
# The lightweight fingerprint cannot prove payload status is unchanged.
fingerprint = fingerprint_library(lmc)
(tmp_path/'payload.pcb').write_bytes(b'native edit without LMC change')
assert fingerprint_library(lmc) == fingerprint
class PendingBridge(FakeBridge):
def query(self, *args, **kwargs):
self.requests.append(args[1])
return BridgeResult('PENDING', 'fake')
bridge = PendingBridge()
records = BatchStatusSession(bridge).scan(['A'], lmc, checkpoint)
assert [record.status for record in records] == ['PENDING']
assert bridge.requests == ['A']
assert len(checkpoint.read_text('utf-8').splitlines()) == 1
def test_timeout_isolated_and_later_item_continues(tmp_path):
lmc = tmp_path / "mentor_lib.lmc"
lmc.write_bytes(b"library")
@@ -180,7 +199,7 @@ def test_retry_rechecks_failed_scan_without_library_changes(tmp_path):
first = BatchStatusSession(FakeBridge(timeout_name='A')).scan(['A', 'B'], lmc, checkpoint)
assert first[0].status == 'TIMEOUT'
bridge = FakeBridge()
second = BatchStatusSession(bridge).scan(['A', 'B'], lmc, checkpoint)
second = BatchStatusSession(bridge).scan(['A', 'B'], lmc, checkpoint, resume=True)
assert bridge.requests == ['A']
assert second[0].status == 'READY'
+1 -1
View File
@@ -82,5 +82,5 @@ def test_fingerprint_change_does_not_reuse_checkpoint(tmp_path):
from cbb_refresh_verify.status_bridge import BridgeResult
return BridgeResult("PENDING", "fake")
bridge = Bridge()
BatchStatusSession(bridge).scan(["A"], lmc, checkpoint)
BatchStatusSession(bridge).scan(["A"], lmc, checkpoint, resume=True)
assert bridge.requests == ["A"]
+1 -1
View File
@@ -269,7 +269,7 @@ def test_status_cache_does_not_reuse_a_different_block_type(tmp_path):
return BridgeResult('READY', 'test')
session = BatchStatusSession(Bridge())
session.scan(['Block'], library, tmp_path/'scan.jsonl', block_type='RULP')
session.scan(['Block'], library, tmp_path/'scan.jsonl', block_type='Other')
session.scan(['Block'], library, tmp_path/'scan.jsonl', block_type='Other', resume=True)
assert calls == ['RULP', 'Other']
-18
View File
@@ -2,7 +2,6 @@ import pytest
from cbb_refresh_verify.pcb_bridge import (
PcbBridgeError,
execution_outcome,
require_document_identity,
wait_for_target_document,
)
@@ -74,23 +73,6 @@ def test_bridge_stops_if_pcb_exits_before_com_is_ready(tmp_path):
assert raised.value.error_code == 'PCB_EXITED_BEFORE_COM'
def test_fatal_dialog_overrides_successful_save_markers():
result = execution_outcome(
save_complete=True,
pcb_exited=True,
fatal_error='SAVE_REUSABLE_BLOCK_FAILED: Error saving the Reusable Block',
)
assert result['status'] == 'FAIL'
assert result['error_code'] == 'SAVE_REUSABLE_BLOCK_FAILED'
def test_success_requires_save_and_pcb_exit():
assert execution_outcome(save_complete=True, pcb_exited=True)['status'] == 'PASS'
assert execution_outcome(save_complete=False, pcb_exited=True)['status'] == 'FAIL'
assert execution_outcome(save_complete=True, pcb_exited=False)['status'] == 'FAIL'
def test_connected_com_without_document_remains_startup_and_checks_native_failure(tmp_path):
errors = iter(('', 'ICDB_REPAIR_REQUIRED: original lost session'))
events = []
+14
View File
@@ -87,6 +87,20 @@ def test_management_records_scoped_changes_without_hashing_unrelated_cbb(setup,
assert 'SymbolLibs/shared/icdb.dat' in report['modified']
assert 'ReusableBlocksLP/Block/Work/Block.prj' in report['modified']
assert result.change_report == report
assert report['rule_version'] == 1
assert report['scope'] == {'kind': 'library', 'block_name': 'Block',
'library_name': request.library.name}
from cbb_refresh_verify import scoped_manifest as manifests
assert report['excluded_categories'] == sorted(manifests._NONPRIMARY_DIRECTORIES)
assert report['excluded_suffixes'] == sorted(manifests._NONPRIMARY_SUFFIXES)
runtime = report['runtime_exclusions']
assert runtime['native_directory_names'] == ['dbcheck', 'dbrestore', 'log']
assert runtime['file_names'] == ['saddress.adr']
assert runtime['file_name_prefixes'] == ['~$']
assert runtime['relative_directories'] == ['projectbackup/backups']
assert runtime['directory_prefixes'] == ['work.before-restore-']
assert 'selected CBB Work/Cert root or its PCB child' in runtime['directory_name_positions']
assert 'SymbolLibs/<partition>/cdbsvr' in runtime['native_directory_positions']
def test_failed_mutation_still_records_changes_and_release_fact(setup):
+31 -6
View File
@@ -118,17 +118,42 @@ def test_corrupted_destination_bytes_are_rejected_and_evidence_is_preserved(tmp_
assert (destination/relative).exists()
def test_unlocked_primary_copy_reads_source_twice_and_target_once(tmp_path, monkeypatch):
from collections import Counter
source, destination = tmp_path/'source', tmp_path/'destination'
primary = put(source, 'A.prj', b'engineering' * 100000)
reads = Counter()
original_open = Path.open
def opened(path, mode='r', *args, **kwargs):
if mode == 'rb':
reads[path] += 1
return original_open(path, mode, *args, **kwargs)
monkeypatch.setattr(Path, 'open', opened)
hashes = Counter()
original_hash = recovery._hash
def hashed(path):
hashes[path] += 1
return original_hash(path)
monkeypatch.setattr(recovery, '_hash', hashed)
manifest = recovery._copy_verified(source, destination)
assert hashes[primary] == 1 # Only the final source rehash is a separate pass.
assert hashes[destination/'A.prj'] == 1
assert reads[primary] == 2 # Copy/hash stream, then final source content proof.
assert reads[destination/'A.prj'] == 1
assert manifest['files']['A.prj']['sha256'] == hashlib.sha256(primary.read_bytes()).hexdigest()
def test_source_byte_changes_with_preserved_metadata_are_rejected(tmp_path, monkeypatch):
source, destination = tmp_path/'source', tmp_path/'destination'
primary = put(source, 'A.prj')
real_copy = shutil.copy2
real_copy = shutil.copystat
def copy_then_change_source(src, dst, *args, **kwargs):
result = real_copy(src, dst, *args, **kwargs)
metadata = primary.stat()
primary.write_bytes(b'X' * metadata.st_size)
os.utime(primary, ns=(metadata.st_atime_ns, metadata.st_mtime_ns))
return result
monkeypatch.setattr(shutil, 'copy2', copy_then_change_source)
monkeypatch.setattr(shutil, 'copystat', copy_then_change_source)
with pytest.raises(OSError):
recovery._copy_verified(source, destination)
assert (destination/'A.prj').read_bytes() == b'engineering'
@@ -140,7 +165,7 @@ def test_complete_tree_metadata_must_stay_stable_during_copy(tmp_path, monkeypat
put(source, 'A.prj')
runtime = put(source, 'logs/session.log')
destination = tmp_path/'destination'
real_copy = shutil.copy2
real_copy = shutil.copystat
def copy_then_mutate(src, dst, *args, **kwargs):
result = real_copy(src, dst, *args, **kwargs)
if Path(src).name == 'A.prj':
@@ -151,7 +176,7 @@ def test_complete_tree_metadata_must_stay_stable_during_copy(tmp_path, monkeypat
else:
put(source, 'logs/added.log')
return result
monkeypatch.setattr(shutil, 'copy2', copy_then_mutate)
monkeypatch.setattr(shutil, 'copystat', copy_then_mutate)
with pytest.raises(OSError):
recovery._copy_verified(source, destination)
assert destination.exists()
@@ -277,7 +302,7 @@ def test_colliding_engineering_names_cannot_bypass_primary_copy_byte_checks(tmp_
scope = ManifestScope.library(block, 'target.lmc')
source, destination = tmp_path/'source', tmp_path/'destination'
put(source, relative)
real_copy = shutil.copy2
real_copy = shutil.copystat
def copy_then_corrupt(src, dst, *args, **kwargs):
result = real_copy(src, dst, *args, **kwargs)
corrupt = Path(src) if side == 'source' else Path(dst)
@@ -285,7 +310,7 @@ def test_colliding_engineering_names_cannot_bypass_primary_copy_byte_checks(tmp_
corrupt.write_bytes(b'X' * metadata.st_size)
os.utime(corrupt, ns=(metadata.st_atime_ns, metadata.st_mtime_ns))
return result
monkeypatch.setattr(shutil, 'copy2', copy_then_corrupt)
monkeypatch.setattr(shutil, 'copystat', copy_then_corrupt)
with pytest.raises(OSError):
recovery._copy_verified(source, destination, scope=scope)
assert (destination/relative).exists()