fix: scope runtime release proof and bound native file probes

This commit is contained in:
阿白叔 committed 2026-11-16 17:37:45 +08:00
1 parent a1b38c242f
commit b070837acb
10 files changed
+816 -77

No files matched your search

@@ -0,0 +1,98 @@
# Task 1 — resource scopes, lease release and bounded proof
Date: 2026-11-16. Worktree: `F:/Sync Tool Skyworth/Work Maps/cbb-refresh-verify-795-issue15`.
Branch: `codex/issue15-runtime-unblock`. Baseline: `efb8cae`.
## Result
Implemented and verified the Task 1 backend changes. The final covering run passed **409 tests in 86.98 seconds**, exit code 0. No production library, native address, user process, or original dirty checkout was modified. No push, PR, full-suite run, frozen build, or real Mentor acceptance was performed by this task.
Release now carries selected single/multiple project scope, so another unselected project's address cannot retain the completed task's sidecar. Unknown checkpoints still require fresh whole-library resource proof, but a process explicitly bound to another library/project is not a global exit requirement. Native inspection-only results are used for exactly mapped addressed databases; addresses remain in place. Native `IN_USE`, unconfirmed/unfinished inspection, real exclusive-file failure, or unreturned shared work reject release.
`data_recovery_required` describes native data diagnostics, independently of lease release. A confirmed NOT_IN_USE database requiring repair does not poison the library or block an explicit subsequent repair/replacement. No persistent recovery blocker store was added, following the controller's product ruling.
## Changes and callers audited
- `library_lock.py`: per-library serialization for entry, exit, maintenance adoption, snapshots and release; `_guard` now protects quick in-memory operations only. Before committing a release, the exact retained-owner object and checkpoint bytes are checked again. Added optional multi-block scope and stop propagation. The upload `.lock` is never deleted.
- `residual_sessions.py`: passes the entire confirmed `block_names` selection into proof, including multi-selection. Existing observation-only rows remain excluded from close actions and active-release counts. Unknown records cannot use an arbitrary selection to narrow their proof.
- `file_probe.py` and static `application.py --file-probe` dispatch: a pure read-only subprocess scans native files and address/PRJ bindings, rejects reparse paths (including junctions within the same library), and acquires real Windows read handles. Default readiness deadline is 10 seconds; stop/timeout terminates only that helper. It never launches native tools. The parent duplicates the held handles before commit, so even the probe's exit cannot create an unprotected interval. Native files exclude other access; address and PRJ handles allow reads but prevent changes/deletion until commit finishes. Local stdin EOF releases helper resources.
- `icdb_ownership.py`: declared session resource scope and exact address-target inspection. An addressed database needs exactly one matching PRJ and `icdb.dat`; ambiguous/shared mappings fail closed. Healthy/no-address projects do not start an extra native inspection.
- `session_record.py`: optional scope/lease/recovery fields on initial session records and results; legacy release receipts without an observed resource scope are rechecked. Observer cleanup requires the same retained alias and operation identity, protecting newer work.
- `native_export_guard.py`: actual retained-export checks remain; explicitly unrelated native owners no longer veto unknown-checkpoint recovery. Export begin/finish use the per-library guard.
- `management_transaction.py`: `current_lease` uses per-library serialization for checkpoint reads. This narrow extension was approved by the controller after the global-lock caller audit.
Audited `_guard` imports/usages in `library_lock`, `native_export_guard`, `management_transaction`, and `native_management.prepare_owned_fixture_add`. The remaining native-management `_guard` block contains only in-memory identity comparisons and handle conversion; its disk reads already occur outside the global lock. Lock order is per-library then short global guard. No UI, queue, `failed_session`, `mentor_adapter`, or `native_backup` edits.
## Downstream interfaces
Public VerifyRequest/VerifyResult are unchanged. Existing positional callers remain valid.
```python
release_retained_library(library, evidence_dir, *, block_name=None, block_names=(), stop=None)
recheck_previous_release(library, evidence_dir, *, block_name=None,
required=False, progress=None, stop=None)
```
Successful release proof retains `status='RELEASE_CONFIRMED'` and adds:
- `lease_state='RELEASED'`, `blocked_resources=[]`.
- `resource_scope`: `scope_state='OBSERVED'`, library, roots, actual project/database/shared-dependency paths, block_name/block_names, lease_id, and reconciled operation_ids.
- `proof_scope`: `BLOCK`, `BLOCKS`, or `LIBRARY`; `proof_block_names` and legacy `proof_block_name` remain available.
- `data_recovery_required`: `None` when no native data diagnosis was needed, otherwise the native boolean.
- `data_recovery_resources`: exact databases whose inspection requires recovery; `native_inspections` contains the full inspection-only records.
- `data_recovery_confirmed=False` remains explicit. Release is not a save/repair/data-health claim.
Initial worker records contain `lease_state='ACTIVE'`, declared resource scope from job `project`/`pcb`, and `data_recovery_required=None`. Missing/legacy project scope is `RECHECK_REQUIRED`. Downstream code should provide actual `project` and `pcb` when known and preserve optional fields in evidence. A subsequent operation must still acquire its own lease and perform its own resource admission; a past proof is not replayable authority.
## TDD evidence
All test/native subprocess invocations used this worktree's absolute `src` in `PYTHONPATH`. No editable install or machine environment modification was used. After the controller's baseline clearance, tests were written and run before their corresponding implementation.
| Red command (after setting absolute PYTHONPATH) | Observed result before fix | Green evidence |
|---|---|---|
| `python -m pytest tests/test_issue15_resource_release.py -q` (initial four tests) | 4 failed in 2.06 s: B's address forced PARTIAL; unrelated process rejected unknown checkpoint; slow A blocked B; old observer cleared new operation | Initial four passed; 83 related tests passed in 10.10 s |
| `python -X utf8 -m pytest tests/test_issue15_resource_release.py -q` (address/helper additions) | 6 failed, 4 passed in 2.08 s: blanket address gate, global unknown export owner gate, missing helper API | Later 151 related tests passed in 26.57 s |
| `... -k 'survives or journal_records or replacement_lease'` | 2 failed, 1 passed in 1.26 s: helper exit dropped proof protection; journal lacked lease scope | Real DuplicateHandle transfer and scoped journal passed in final run |
| `... -k 'multiple_selected or without_release'` | 1 failed, 3 passed in 2.46 s: A+B expanded into C | Multi-block regression passed; IN_USE/UNCONFIRMED/not-exited safety tests retained |
| `.../run_tests.py -q tests/test_issue15_resource_release.py -k 'legacy_release or stalled_transaction'` | 2 failed in 1.29 s: legacy receipt trusted; stalled checkpoint held global guard | 2 passed in 0.33 s |
| `... -k 'binding_against'` | 1 failed in 1.02 s: address/PRJ writable during proof | 1 passed in 0.62 s |
| `... -k 'scope_junction'` | 1 failed in 0.99 s: same-library junction silently changed target | Passed in final run |
| `... -k 'legacy_release'` (malformed scope case) | 1 failed, 1 passed in 0.76 s: null old scope raised AttributeError | 2 passed in 0.27 s |
The regression module has 22 cases. Real Windows checks include parent-held sidecar rejecting a child then succeeding after scoped release; duplicated exclusive handles surviving probe termination; actual write rejection on address/project binding files; real sleeping subprocess timeout/cancellation and reaping; and an actual Windows junction. Native inspection output is mocked at the native boundary with complete exact-target records; no actual Mentor was launched by these tests.
Additional related runs: 239 passed in 48.24 s (maintenance/export/management/packaging); 214 passed in 38.77 s after global-lock and legacy receipt changes. A test invocation containing nonexistent `tests/test_management_transaction.py` collected no tests; it was corrected to the actual `test_management_native_transaction.py` in the final run. An intermediate external-owner regression exposed loss of WinError 32 through the helper; error propagation was restored and all final cases pass.
## Final verification
Durable local artifacts:
- `.superpowers/sdd/2026-11-16-issue15-runtime-unblock/task-1-tests.log` — exact command and complete pytest output.
- `.superpowers/sdd/2026-11-16-issue15-runtime-unblock/task-1-tests.xml` — JUnit results.
The covering command used the controller's runner, which asserts canonical imports and sets inherited absolute PYTHONPATH:
```text
python -X utf8 .superpowers/sdd/2026-11-16-issue15-runtime-unblock/run_tests.py -q
--junitxml=.superpowers/sdd/2026-11-16-issue15-runtime-unblock/task-1-tests.xml
tests/test_issue15_resource_release.py tests/test_library_lock.py
tests/test_session_record.py tests/test_residual_sessions.py
tests/test_external_owner_release.py tests/test_review_release_followup.py
tests/test_review_foreign_release_evidence.py tests/test_operation_lifecycle.py
tests/test_owner_probe.py tests/test_native_export_guard.py
tests/test_management_native_transaction.py tests/test_management_retry_blockers.py
tests/test_management_probe_guards.py tests/test_independent_operations.py
tests/test_maintenance_entry.py tests/test_maintenance_owner_scope.py
tests/test_maintenance_review.py tests/test_review_backend_followup.py
tests/test_review_319.py tests/test_review326_dispatch_and_receipts.py
tests/test_packaging.py tests/test_packaging_status_bridge.py
```
Actual summary: `409 passed in 86.98s (0:01:26)`; exit code 0. `git diff --check` also passed.
## Concerns and acceptance boundaries
- **Downstream Task 2/4 requirement:** The 10-second cancellable helper bounds native-tree/address/PRJ discovery and native exclusive-file acquisition. Own checkpoint read/write/archive and existing retained-export evidence paths remain synchronous, but are now per-library and do not hold the global memory guard. A wedged network/filesystem operation in those existing paths can still delay that library. Task 2 must apply deadline/handoff and late-call cancellation guards so this cannot retain the global queue/UI indefinitely; Task 4 must verify it. This is not a whole-call timeout claim. The controller explicitly kept that lifecycle work out of Task 1.
- Native inspection runs through the existing independent inspection-only supervisor and is never killed by the pure file-probe timeout. Its diagnostics use retained local temporary directories; complete native result records are also embedded in release evidence. Native tool behavior, native close timing, and on-site reproduction remain unverified here.
- Ambiguous/shared addresses without an exact unique PRJ-to-database mapping remain protected for manual investigation. No address age, unreachable endpoint, process absence alone, or metadata PID is accepted as ownership/release authority.
- No automatic Repair or automatic whole-library backup was added. Full application cancellation/batch/shutdown policy and UI use of the added fields belong to subsequent tasks. Frozen packaging was covered by existing packaging tests and static helper dispatch import, not a new EXE build.
+4
View File
@@ -77,6 +77,7 @@ def main(argv: Sequence[str] | None = None) -> int:
parser.add_argument('--verify-worker', type=Path, help=argparse.SUPPRESS)
parser.add_argument('--repair-worker', type=Path, help=argparse.SUPPRESS)
parser.add_argument('--owner-probe', type=Path, help=argparse.SUPPRESS)
parser.add_argument('--file-probe', type=Path, help=argparse.SUPPRESS)
parser.add_argument('--native-backup-worker', type=Path, help=argparse.SUPPRESS)
parser.add_argument('--management-worker', type=Path, help=argparse.SUPPRESS)
parser.add_argument('--dialog-agent', type=Path, help=argparse.SUPPRESS)
@@ -89,6 +90,9 @@ def main(argv: Sequence[str] | None = None) -> int:
parser.add_argument('--mentor-bin', default='', help=argparse.SUPPRESS)
parser.add_argument("--smoke-test", action="store_true", help=argparse.SUPPRESS); parser.add_argument("--data-root", type=Path, help=argparse.SUPPRESS)
args = parser.parse_args(list(argv) if argv is not None else None)
if args.file_probe:
from .file_probe import file_probe_main
return file_probe_main(args.file_probe)
if args.owner_probe:
from .owner_probe import owner_probe_main
return owner_probe_main(args.owner_probe)
+151
View File
@@ -0,0 +1,151 @@
"""Bound read-only filesystem probes; keep proof handles alive through commit."""
from contextlib import contextmanager, ExitStack
import hashlib
import json
from pathlib import Path
import re
import stat
import subprocess
import sys
import tempfile
import time
from uuid import uuid4
@contextmanager
def hold_release_files(library, roots=None, *, timeout_seconds=10, stop=None):
"""Only this pure probe may be terminated. stdin EOF releases its handles."""
with tempfile.TemporaryDirectory(prefix='cbb-file-probe-') as folder:
job = Path(folder)/'request.json'
token = uuid4().hex
job.write_text(json.dumps({'token': token, 'library': str(library),
'roots': [str(p) for p in roots] if roots is not None else None}), 'utf-8')
command = ([sys.executable] if getattr(sys, 'frozen', False) else
[sys.executable, '-m', 'cbb_refresh_verify']) + ['--file-probe', str(job)]
process = subprocess.Popen(command, stdin=subprocess.PIPE, stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL, creationflags=getattr(subprocess, 'CREATE_NO_WINDOW', 0))
try:
deadline = time.monotonic()+max(.1, min(10, timeout_seconds))
result_path = job.with_name('result.json')
while not result_path.exists():
if stop is not None and stop.is_set():
raise InterruptedError('FILE_PROBE_CANCELLED')
if time.monotonic() >= deadline:
raise TimeoutError('FILE_PROBE_TIMEOUT: 文件占用检查超时,未证明释放')
if process.poll() is not None:
raise OSError('FILE_PROBE_UNCONFIRMED: 文件探测未返回证据')
time.sleep(.02)
if stop is not None and stop.is_set():
raise InterruptedError('FILE_PROBE_CANCELLED')
record = json.loads(result_path.read_text('utf-8'))
if record.get('token') != token or record.get('status') != 'HELD' or process.poll() is not None:
error = OSError('FILE_PROBE_UNCONFIRMED: '+str(record.get('error', '探测身份或句柄失效')))
error.winerror = record.get('winerror')
raise error
# Duplicate the already-open handles before acknowledging the probe.
# Its exit can never create a gap in the parent's commit protection.
import win32api, win32con
with ExitStack() as protection:
child = win32api.OpenProcess(win32con.PROCESS_DUP_HANDLE, False, process.pid)
protection.callback(child.Close)
if len(record['handles']) != len(record['exclusive_files'])+len(record['binding_files']):
raise OSError('FILE_PROBE_UNCONFIRMED: 证明句柄数量不符')
for handle in record['handles']:
owned = win32api.DuplicateHandle(child, handle, win32api.GetCurrentProcess(),
0, False, win32con.DUPLICATE_SAME_ACCESS)
protection.callback(owned.Close)
yield {k: v for k, v in record.items() if k != 'handles'}
finally:
process.stdin.close()
try:
process.wait(.5)
except subprocess.TimeoutExpired:
process.terminate() # This executable only scans and opens read handles.
process.wait(3)
def _snapshot(library, roots):
from .scoped_manifest import scan_tree
entries = {library: library.lstat()}
for root in roots:
if root.exists():
entries.update(scan_tree(root))
native = sorted(path for path, info in entries.items() if stat.S_ISREG(info.st_mode)
and (path == library or path.name.casefold() in ('icdb.dat', 'layoutdb.lyt')))
if library not in native:
raise ValueError('目标库身份缺失')
addresses = sorted(path for path in entries if path.name.casefold() == 'saddress.adr')
targets, signatures = [], {}
for address in addresses:
data = address.read_bytes()
signatures[str(address)] = hashlib.sha256(data).hexdigest()
database = address.parent.parent
matches = []
for project in database.parent.glob('*.prj'):
if project not in entries or project.stat().st_size > 1024*1024:
continue
from .backup_recovery import _project_text
text = _project_text(project)
references = re.findall(r'^KEY iCDBDir "([^"\r\n]+)"', text, re.M)
if len(references) == 1 and (project.parent/references[0]).resolve() == database:
matches.append(project)
if len(matches) != 1 or database/'icdb.dat' not in native:
raise ValueError(f'地址未绑定唯一工程数据库,需人工核对:{address}')
targets.append({'project': str(matches[0]), 'database': str(database)})
signatures[str(matches[0])] = hashlib.sha256(matches[0].read_bytes()).hexdigest()
return {'exclusive_files': [str(p) for p in native], 'addresses': [str(p) for p in addresses],
'inspection_targets': targets, 'signatures': signatures, 'binding_files': sorted(signatures),
'resource_scope': {'library': str(library), 'roots': [str(p) for p in roots],
'projects': sorted(str(p) for p in entries if p.suffix.casefold() == '.prj'),
'shared_dependencies': sorted(str(p.parent) for p in native
if p.is_relative_to(library.parent/'SymbolLibs') and p.name.casefold() == 'icdb.dat'),
'databases': sorted({str(p.parent) for p in native if p.name.casefold() == 'icdb.dat'})}}
def file_probe_main(job):
"""No native launch, writes, repair, address deletion or process termination."""
import win32con, win32file
job = Path(job)
request = json.loads(job.read_text('utf-8'))
result = {'token': request['token'], 'status': 'UNCONFIRMED'}
try:
def checked_path(value):
path = Path(value).absolute()
for part in (path, *path.parents):
try:
info = part.lstat()
except FileNotFoundError:
continue
if info.st_file_attributes & stat.FILE_ATTRIBUTE_REPARSE_POINT:
raise ValueError(f'资源范围含重解析点:{part}')
return path.resolve()
library = checked_path(request['library'])
roots = [checked_path(p) for p in request['roots']] if request['roots'] is not None else [library.parent]
if library.suffix.casefold() != '.lmc' or any(not p.is_relative_to(library.parent) for p in roots):
raise ValueError('文件探测范围不属于所选库')
snapshot = _snapshot(library, roots)
with ExitStack() as stack:
handles = []
for path in snapshot['exclusive_files']:
handle = win32file.CreateFile(path, win32con.GENERIC_READ, 0, None,
win32con.OPEN_EXISTING, win32con.FILE_ATTRIBUTE_NORMAL, None)
stack.callback(handle.Close)
handles.append(int(handle))
for path in snapshot['binding_files']:
handle = win32file.CreateFile(path, win32con.GENERIC_READ, win32con.FILE_SHARE_READ,
None, win32con.OPEN_EXISTING, win32con.FILE_ATTRIBUTE_NORMAL, None)
stack.callback(handle.Close)
handles.append(int(handle))
if _snapshot(library, roots) != snapshot:
raise ValueError('占用检查期间资源集合变化')
pending = job.with_name('result.pending')
pending.write_text(json.dumps({**result, **snapshot, 'status': 'HELD', 'handles': handles}), 'utf-8')
pending.replace(job.with_name('result.json'))
sys.stdin.buffer.read(1) # Parent keeps stdin open until its commit finishes.
return 0
except Exception as error:
pending = job.with_name('result.pending')
pending.write_text(json.dumps({**result, 'error': str(error),
'winerror': getattr(error, 'winerror', None)}), 'utf-8')
pending.replace(job.with_name('result.json'))
return 1
+30
View File
@@ -2,6 +2,19 @@
from pathlib import Path
def session_resource_scope(job):
"""Persist declared targets; only fresh proof promotes them to observed scope."""
library = Path(job['library'])
project, pcb = job.get('project'), job.get('pcb')
projects = [str(Path(project))] if project else []
databases = [str(Path(p).parent/'database') for p in (project, pcb) if p]
shared = library.parent/'SymbolLibs/__ReusableBlocksSymb'
return {'scope_state': 'DECLARED' if project else 'RECHECK_REQUIRED',
'library': str(library), 'operation_id': job.get('operation_id'),
'block_name': job.get('block'), 'projects': projects,
'databases': list(dict.fromkeys(databases)), 'shared_dependencies': [str(shared)]}
def database_directories(lmc_file, project, pcb):
return tuple(dict.fromkeys((
Path(project).parent/'database',
@@ -14,3 +27,20 @@ def unreleased_addresses(lmc_file, project, pcb):
return [str(path/'cdbsvr/sAddress.adr')
for path in database_directories(lmc_file, project, pcb)
if (path/'cdbsvr/sAddress.adr').exists()]
def inspect_release_addresses(snapshot, *, stop=None):
"""An address is a reason for native inspection, never an occupancy verdict."""
import tempfile
from .native_repair import RepairTarget, run_native_repair
from .native_occupancy import confirmed_inspection
from .library_lock import LibraryBusy
results = []
for item in snapshot['inspection_targets']:
target = RepairTarget(Path(item['project']), Path(item['database']))
root = Path(tempfile.mkdtemp(prefix='cbb-release-inspection-'))
result = run_native_repair(target, root, inspection_only=True, stop=stop)
if not confirmed_inspection(target, result) or result['status'] != 'NOT_IN_USE':
raise LibraryBusy(f'目标数据库未确认释放:{target.database};检查证据:{root}')
results.append(result)
return results
+105 -61
View File
@@ -11,6 +11,13 @@ from uuid import uuid4
_guard = threading.RLock()
_owners = {}
_library_guards = {}
def _library_guard(key):
"""Serialize one lease without making slow resource probes global."""
with _guard:
return _library_guards.setdefault(key, threading.RLock())
class LibraryBusy(OSError):
@@ -82,10 +89,10 @@ def has_retained_library(library):
return bool(owner and owner['retained'])
def recheck_previous_release(library, evidence_dir, *, block_name=None, required=False, progress=None):
def recheck_previous_release(library, evidence_dir, *, block_name=None, required=False, progress=None, stop=None):
"""Historical records trigger fresh proof, never process termination or repair."""
if not required:
with _guard:
with _library_guard(LibraryLock(library).key):
owner = _owners.get(LibraryLock(library).key)
marker = blocker_path(library)
record = checkpoint_record(marker)
@@ -108,7 +115,8 @@ def recheck_previous_release(library, evidence_dir, *, block_name=None, required
logging.getLogger(__name__).warning('Release progress observer failed', exc_info=True)
notify('previous_release_check', detail='正在复核历史记录对应的实际占用')
try:
proof = release_retained_library(library, evidence_dir, block_name=block_name)
options = {'stop': stop} if stop is not None else {}
proof = release_retained_library(library, evidence_dir, block_name=block_name, **options)
except Exception as error:
notify('previous_release_unconfirmed', detail=str(error))
raise
@@ -128,9 +136,14 @@ def _persist_blocker(library, status='OWNER_RELEASE_UNCONFIRMED', *, lease_id, b
export_record['block_name'] = block_name
elif previous and previous['lease_id'] == lease_id and 'block_name' in previous:
export_record['block_name'] = previous['block_name']
from .icdb_ownership import session_resource_scope
scope = (previous.get('resource_scope') if previous and previous['lease_id'] == lease_id else None)
scope = scope or session_resource_scope({'library': library, 'block': export_record.get('block_name')})
with pending.open('w', encoding='utf-8') as stream:
json.dump({**export_record, 'protocol': 'cbb-refresh-verify-library-lease', 'version': 1,
'lease_id': lease_id, 'library': str(library), 'status': status,
'lease_state': 'ACTIVE' if status == 'OPERATION_IN_PROGRESS' else 'RETAINED',
'resource_scope': scope, 'blocked_resources': [], 'data_recovery_required': None,
'machine': socket.gethostname(),
'supervisor_created': win32process.GetProcessTimes(win32api.GetCurrentProcess())['CreationTime'].isoformat(),
'pid': os.getpid(), 'time': datetime.now(timezone.utc).isoformat()}, stream)
@@ -159,7 +172,7 @@ class LibraryLock:
return lease
def __enter__(self):
with _guard:
with _library_guard(self.key):
owner = _owners.get(self.key)
if owner and owner['retained'] and owner['depth'] == 0 and owner.get('release_probe'):
try:
@@ -180,7 +193,7 @@ class LibraryLock:
if self.maintenance_guard is not None:
return _enter_maintenance(self)
import win32con, win32file
with _guard:
with _library_guard(self.key):
owner = _owners.get(self.key)
if owner:
if owner['thread'] != threading.get_ident() or owner['retained']:
@@ -246,7 +259,7 @@ class LibraryLock:
self.maintenance_mutex = None
def _exit_owned(self):
with _guard:
with _library_guard(self.key):
if not self.entered:
return
owner = _owners[self.key]
@@ -402,7 +415,7 @@ def _enter_maintenance(lease):
try:
acquired = win32event.WaitForSingleObject(mutex, 0) in (win32con.WAIT_OBJECT_0, win32con.WAIT_ABANDONED)
if not acquired: raise LibraryBusy('已有 CBB 操作运行,维护不能接续')
with _guard:
with _library_guard(lease.key):
marker = blocker_path(lease.library)
if marker.with_suffix('.pending').exists():
raise LibraryBusy('未完成检查点写入,不能自动接续')
@@ -455,7 +468,7 @@ def _enter_maintenance(lease):
def owned_snapshot(path):
"""Read only a lease held by this thread, never reopen an exclusive file."""
import win32con, win32file
with _guard:
with _library_guard(_key(path)):
owner = _owners.get(_key(path))
if not owner or owner['thread'] != threading.get_ident() or owner['retained']:
return None
@@ -470,19 +483,16 @@ def owned_snapshot(path):
@contextmanager
def _prove_library_released(library, block_name=None, *, allow_unknown_checkpoint=False):
def _prove_library_released(library, block_name=None, *, block_names=(), allow_unknown_checkpoint=False, stop=None):
"""Hold all native database read handles during explicit lease release."""
import stat
import win32con, win32file
from .owner_probe import query_owners
from .scoped_manifest import scan_tree
from .native_management import _retained_workers
from .native_export_guard import assert_retained_exports_idle, prove_export_copies_released
from .native_engineering_identity import _OWNER_NAMES
library = Path(library).resolve()
assert_retained_exports_idle(library)
if any(worker.poll() is None and (allow_unknown_checkpoint
or not getattr(worker, 'cbb_library', None) or _key(worker.cbb_library) == _key(library))
if any(worker.poll() is None and (not getattr(worker, 'cbb_library', None)
or _key(worker.cbb_library) == _key(library))
for worker in _retained_workers):
raise LibraryBusy('原生管理工作进程尚未返回,不能解除协调阻止。')
from .com_runtime import initialize_com, uninitialize_com
@@ -492,12 +502,17 @@ def _prove_library_released(library, block_name=None, *, allow_unknown_checkpoin
'iCDBProjectBackup.exe', 'ReuseBlockImporter.exe', 'CleanWGLib.exe',
'cscript.exe', 'wscript.exe', 'python.exe', 'pythonw.exe',
'CbbRefreshVerify.exe', *sorted(_OWNER_NAMES))
scope_root = (Path(library).resolve().parent/'ReusableBlocksLP'/block_name/'Work').resolve() if block_name else None
blocks = tuple(dict.fromkeys(block_names or ((block_name,) if block_name else ())))
if any(not name or name in ('.', '..') or any(c in name for c in '\\/:*?"<>|') for name in blocks):
raise ValueError('释放检查的工程名称无效')
scope_roots = tuple(library.parent/'ReusableBlocksLP'/name/'Work' for name in blocks)
shared_root = Path(library).resolve().parent/'SymbolLibs'
if allow_unknown_checkpoint and scope_root is not None:
if allow_unknown_checkpoint and scope_roots:
raise LibraryBusy('未知检查点必须执行完整库释放证明。')
observed = []
def check_owners():
if stop is not None and stop.is_set():
raise InterruptedError('已停止释放检查')
owners = []
observed.clear()
from .session_record import active_write_records
@@ -506,12 +521,11 @@ def _prove_library_released(library, block_name=None, *, allow_unknown_checkpoin
for item in query_owners(names):
if item['pid'] == os.getpid():
continue
if not allow_unknown_checkpoint:
roots = (scope_root, shared_root) if scope_root is not None else (Path(library).parent,)
if not _owner_is_blocking(item, library, roots):
if _owner_matches_scope(item, library, roots) is None:
observed.append(item)
continue
roots = (*scope_roots, shared_root) if scope_roots else (library.parent,)
if not _owner_is_blocking(item, library, roots):
if _owner_matches_scope(item, library, roots) is None:
observed.append(item)
continue
automation = item['name'].casefold() in ('python.exe', 'pythonw.exe', 'cbbrefreshverify.exe')
command = item['command'].casefold()
if not automation or not command or any(flag in command for flag in (
@@ -521,44 +535,48 @@ def _prove_library_released(library, block_name=None, *, allow_unknown_checkpoin
raise LibraryBusy(f'原生进程仍存在,不能解除协调阻止:{owners}')
return owners
check_owners()
if scope_root is None:
entries = scan_tree(Path(library).parent)
else:
entries = {Path(library): Path(library).lstat()}
for root in (scope_root, shared_root):
if root.exists():
entries.update(scan_tree(root))
addresses = [str(path) for path in entries if path.name.casefold() == 'saddress.adr'
and (scope_root is None or path.is_relative_to(scope_root)
or path.is_relative_to(shared_root))]
if addresses:
raise LibraryBusy(f'库中仍有数据库地址记录,不能自动解除:{addresses}')
# scan_tree already rejects links; only native candidates need file opens.
native = [path for path, metadata in entries.items() if stat.S_ISREG(metadata.st_mode) and (
path == library
or (path.name.casefold() in ('icdb.dat', 'layoutdb.lyt')
and (scope_root is None or path.is_relative_to(scope_root)
or path.is_relative_to(shared_root))))]
if library not in native:
raise LibraryBusy('目标库身份缺失,不能解除协调阻止。')
from .file_probe import hold_release_files
roots = (*scope_roots, shared_root) if scope_roots else None
inspections = []
with ExitStack() as stack:
try:
files = stack.enter_context(hold_release_files(library, roots, stop=stop))
except OSError as error:
blocked = LibraryBusy(str(error))
blocked.winerror = getattr(error, 'winerror', None)
raise blocked from error
if files['addresses']:
# Native inspection needs the database; reacquire proof handles
# and compare the exact address/project binding before commit.
stack.close()
from .icdb_ownership import inspect_release_addresses
inspections = inspect_release_addresses(files, stop=stop)
try:
current = stack.enter_context(hold_release_files(library, roots, stop=stop))
except OSError as error:
blocked = LibraryBusy(str(error))
blocked.winerror = getattr(error, 'winerror', None)
raise blocked from error
if any(current[key] != files[key] for key in (
'exclusive_files', 'addresses', 'signatures', 'inspection_targets')):
raise LibraryBusy('原生检查期间资源范围或地址身份变化。')
files = current
export_proof = (prove_export_copies_released(library, allow_unknown_checkpoint=True)
if allow_unknown_checkpoint else prove_export_copies_released(library))
export_copies = stack.enter_context(export_proof)
for path in native:
handle = win32file.CreateFile(str(path), win32con.GENERIC_READ, 0, None,
win32con.OPEN_EXISTING, win32con.FILE_ATTRIBUTE_NORMAL, None)
stack.callback(handle.Close)
owners = check_owners()
yield {'native_owners': owners, 'observed_owners': list(observed),
'addresses': addresses, 'export_copies': export_copies,
'unknown_checkpoint_recovery': allow_unknown_checkpoint,
'exclusive_files': [str(path) for path in native]}
yield {**{k: v for k, v in files.items() if k not in ('status', 'token')},
'native_owners': owners, 'observed_owners': list(observed),
'export_copies': export_copies, 'native_inspections': inspections,
'data_recovery_required': any(item['repair_required'] for item in inspections) if inspections else None,
'data_recovery_resources': [item['database'] for item in inspections if item['repair_required']],
'unknown_checkpoint_recovery': allow_unknown_checkpoint}
finally:
uninitialize_com(initialized)
def release_retained_library(library, evidence_dir, *, block_name=None):
def release_retained_library(library, evidence_dir, *, block_name=None, block_names=(), stop=None):
"""Reconcile historical leases after fresh owner/address/access proof.
The upload sidecar is never deleted. This acknowledges process release,
@@ -569,6 +587,9 @@ def release_retained_library(library, evidence_dir, *, block_name=None):
from .session_record import (retained_session_snapshot, acknowledge_library_release,
reconciled_retained_sessions)
retained = retained_session_snapshot(library)
requested_blocks = tuple(dict.fromkeys(block_names or ((block_name,) if block_name else ())))
if any(not name or name in ('.', '..') or any(c in name for c in '\\/:*?"<>|') for name in requested_blocks):
raise ValueError('释放检查的工程名称无效')
evidence = Path(evidence_dir).resolve()
if evidence.is_relative_to(library.parent) or library.parent.is_relative_to(evidence):
raise ValueError('释放检查证据必须位于目标库之外。')
@@ -577,10 +598,12 @@ def release_retained_library(library, evidence_dir, *, block_name=None):
acquired = False
external = None
try:
if stop is not None and stop.is_set():
raise InterruptedError('已停止释放检查')
acquired = win32event.WaitForSingleObject(mutex, 0) in (win32con.WAIT_OBJECT_0, win32con.WAIT_ABANDONED)
if not acquired:
raise LibraryBusy('另一个 CBB 操作仍在运行。')
with _guard:
with _library_guard(lease.key):
owner = _owners.get(lease.key)
if owner and (not owner['retained'] or owner['depth']):
raise LibraryBusy('本应用操作仍在使用库锁。')
@@ -598,8 +621,10 @@ def release_retained_library(library, evidence_dir, *, block_name=None):
before = {path: path.read_bytes() if path.exists() else None for path in checkpoints}
unknown = any(data is not None and checkpoint_record(path) is None for path, data in before.items())
records = [checkpoint_record(path) for path, data in before.items() if data is not None]
scoped = (block_name if block_name and all(record and record.get('block_name') == block_name
for record in records) and (owner is None or owner.get('block_name') == block_name) else None)
scoped_blocks = (requested_blocks if requested_blocks and all(record
and record.get('block_name') in requested_blocks for record in records)
and (owner is None or owner.get('block_name') in requested_blocks) else ())
scoped = scoped_blocks[0] if len(scoped_blocks) == 1 else None
reconciled = [{'kind': 'checkpoint', 'path': str(path), 'recognized': metadata is not None,
**{key: (metadata or {}).get(key) for key in ('block_name', 'lease_id', 'pid', 'status', 'time')}}
for path, data in before.items() if data is not None
@@ -610,27 +635,46 @@ def release_retained_library(library, evidence_dir, *, block_name=None):
'status': 'OWNER_RELEASE_UNCONFIRMED'})
foreign = sorted({item['block_name'] for item in reconciled
if block_name and item.get('block_name') and item['block_name'] != block_name})
proof_context = (_prove_library_released(library, allow_unknown_checkpoint=True) if unknown else
_prove_library_released(library, block_name=scoped)
if scoped else _prove_library_released(library))
options = {'stop': stop} if stop is not None else {}
if len(scoped_blocks) > 1:
options['block_names'] = scoped_blocks
proof_context = (_prove_library_released(library, allow_unknown_checkpoint=True, **options) if unknown else
_prove_library_released(library, block_name=scoped, **options)
if scoped else _prove_library_released(library, **options))
with proof_context as proof:
if stop is not None and stop.is_set():
raise InterruptedError('已停止释放检查')
with _guard:
if _owners.get(lease.key) is not owner:
raise LibraryBusy('释放检查期间租约身份变化,保留当前任务。')
if unknown and proof.get('unknown_checkpoint_recovery') is not True:
raise LibraryBusy('未知检查点缺少完整库释放证明,保留原文件。')
if any((path.read_bytes() if path.exists() else None) != data for path, data in before.items()):
raise LibraryBusy('释放检查期间协调检查点变化,保留原文件。')
root = evidence/('release-' + uuid4().hex)
root.mkdir(parents=True, exist_ok=False)
reconciled_sessions = reconciled_retained_sessions(retained, block_name=scoped)
matching_retained = ({path: value for path, value in retained.items()
if value[1] in scoped_blocks} if scoped_blocks else retained)
reconciled_sessions = reconciled_retained_sessions(matching_retained, block_name=scoped)
reconciled_blocks = sorted({item['block_name'] for item in (*reconciled, *reconciled_sessions)
if item.get('block_name')})
record = {'library': str(library), 'status': 'RELEASE_CONFIRMED', **proof,
'requested_block_name': block_name, 'proof_scope': 'BLOCK' if scoped else 'LIBRARY',
'proof_block_name': scoped, 'reconciled_checkpoints': reconciled,
'requested_block_name': block_name, 'requested_block_names': list(requested_blocks),
'proof_scope': 'BLOCK' if scoped else 'BLOCKS' if scoped_blocks else 'LIBRARY',
'proof_block_name': scoped, 'proof_block_names': list(scoped_blocks),
'reconciled_checkpoints': reconciled,
'reconciled_foreign_blockers': sorted(set(foreign) | {
block for block in reconciled_blocks if block_name and block != block_name}),
'reconciled_blocks': reconciled_blocks, 'reconciled_sessions': reconciled_sessions,
'unknown_checkpoint_archived': unknown,
'lease_state': 'RELEASED', 'blocked_resources': [],
'data_recovery_required': proof.get('data_recovery_required', None),
'data_recovery_confirmed': False, 'evidence_path': str(root/'release.json')}
record['resource_scope'] = {**proof.get('resource_scope', {}),
'scope_state': 'OBSERVED', 'library': str(library), 'block_name': scoped,
'block_names': list(scoped_blocks),
'lease_id': owner['lease_id'] if owner else None,
'operation_ids': [item['operation_id'] for item in reconciled_sessions]}
(root/'release.json').write_text(json.dumps(record, ensure_ascii=False, indent=2), encoding='utf-8')
for index, checkpoint in enumerate(checkpoints):
if checkpoint.exists():
@@ -643,7 +687,7 @@ def release_retained_library(library, evidence_dir, *, block_name=None):
if owner:
owner['handle'].Close()
del _owners[lease.key]
acknowledge_library_release(record, retained, block_name=scoped)
acknowledge_library_release(record, matching_retained, block_name=scoped)
return record
finally:
if external is not None:
@@ -25,9 +25,9 @@ def fixture_authority(root, library, token):
def current_lease(library):
from .library_lock import LibraryLock, _guard, _owners, blocker_path, checkpoint_record
from .library_lock import LibraryLock, _library_guard, _owners, blocker_path, checkpoint_record
lease = LibraryLock(library)
with _guard:
with _library_guard(lease.key):
owner = _owners.get(lease.key)
guard = checkpoint_record(blocker_path(library))
if (owner is None or owner['retained'] or owner['thread'] != threading.get_ident()
@@ -79,7 +79,7 @@ def assert_retained_exports_idle(library=None):
def begin_export_analysis(library, root, project):
from .library_lock import _guard, blocker_path, checkpoint_record, LibraryBusy
from .library_lock import LibraryLock, _library_guard, blocker_path, checkpoint_record, LibraryBusy
from .management_transaction import current_lease
library, root, project = Path(library).resolve(), Path(root).resolve(), Path(project).resolve()
owner = current_lease(library)
@@ -89,7 +89,7 @@ def begin_export_analysis(library, root, project):
record = {'protocol': 'cbb-native-export-analysis', 'version': 1, 'token': uuid4().hex,
'library': str(library), 'lease_id': owner['lease_id'], 'supervisor_pid': owner['supervisor_pid'],
'analysis_root': str(root), 'project': str(project)}
with _guard:
with _library_guard(LibraryLock(library).key):
marker = blocker_path(library)
checkpoint = checkpoint_record(marker)
if (checkpoint is None or checkpoint['lease_id'] != owner['lease_id']
@@ -116,8 +116,10 @@ def prove_export_copies_released(library, *, allow_unknown_checkpoint=False, uns
raise LibraryBusy('Invalid library checkpoint cannot release export ownership.')
# Only the explicit release path supplies full-library/native proof and
# archives the unchanged metadata while its coordination handle is held.
assert_retained_exports_idle()
if any(p['name'].casefold() in _OWNER_NAMES for p in _process_snapshot()):
from .library_lock import _owner_matches_scope
if any(p['name'].casefold() in _OWNER_NAMES and
_owner_matches_scope(p, library, (library.parent,)) is not False
for p in _process_snapshot()):
raise LibraryBusy('Unknown export checkpoint still has a possible native identity owner.')
yield []
return
@@ -208,8 +210,8 @@ def prove_export_copies_released(library, *, allow_unknown_checkpoint=False, uns
def finish_export_analysis(library, token, *, native_not_started=False):
from .library_lock import _guard, blocker_path, checkpoint_record, LibraryBusy
with _guard, prove_export_copies_released(library,
from .library_lock import LibraryLock, _library_guard, blocker_path, checkpoint_record, LibraryBusy
with _library_guard(LibraryLock(library).key), prove_export_copies_released(library,
unstarted_token=token if native_not_started else ''):
marker = blocker_path(library)
checkpoint = checkpoint_record(marker)
+6 -3
View File
@@ -560,9 +560,11 @@ def _force_close(row, *, stop=None):
'detail': '已强制结束客户端,工程待检查,未执行修复' if exited else '已请求结束客户端,等待退出'}
def _finish_library_release(library, evidence):
def _finish_library_release(library, evidence, block_names=(), stop=None):
from .library_lock import release_retained_library
return release_retained_library(library, evidence)
return release_retained_library(library, evidence,
block_name=block_names[0] if len(block_names) == 1 else None,
block_names=block_names, stop=stop)
def _request_native_cancel(row, *, stop=None):
@@ -713,7 +715,8 @@ def _release_residual_sessions(plan, evidence_dir, *, stop=None, force=False, ca
if row.get('force_user_confirmed') is True else None),
native_activity=native_activity(row['name'], windows, incomplete=error))
if release_targets_complete(result['sessions']) and not (stop and stop.is_set()):
result['proof'] = _finish_library_release(library, root/'library-proof')
result['proof'] = _finish_library_release(library, root/'library-proof',
blocks, stop)
proof = result['proof']
if (proof.get('status') != 'RELEASE_CONFIRMED'
or Path(proof.get('library', '')).resolve() != library):
+24 -5
View File
@@ -108,9 +108,12 @@ def library_wait_reason(library):
def register_worker_session(worker, job):
"""The launcher must persist identity before opening the worker's gate."""
path = Path(job['session_record_path'])
from .icdb_ownership import session_resource_scope
write_session_record(path, {k: job.get(k) for k in (
'operation_id', 'operation', 'library', 'block', 'desktop', 'cancel_signal')}
| {'processes': [{'role': 'worker', **process_identity(worker.pid)}]})
| {'processes': [{'role': 'worker', **process_identity(worker.pid)}],
'lease_state': 'ACTIVE', 'resource_scope': session_resource_scope(job),
'blocked_resources': [], 'data_recovery_required': None})
worker.session_record_path = path
worker.cbb_library = job['library']
@@ -230,7 +233,13 @@ def _result(status, detail='', *, session=None, state=None, proof=None):
return {'status': status, 'detail': detail, 'library': (session or {}).get('library'),
'block': (session or {}).get('block'),
'operation_id': (session or {}).get('operation_id'),
'phase': (state or {}).get('phase', 'UNCONFIRMED'), 'release_proof': proof}
'phase': (state or {}).get('phase', 'UNCONFIRMED'), 'release_proof': proof,
'lease_state': (proof or {}).get('lease_state', 'ACTIVE' if status == 'ACTIVE' else 'UNCONFIRMED'),
'resource_scope': (proof or {}).get('resource_scope') or (session or {}).get('resource_scope')
or {'scope_state': 'RECHECK_REQUIRED'},
'blocked_resources': (proof or {}).get('blocked_resources', (session or {}).get('blocked_resources', [])),
'data_recovery_required': (proof or {}).get('data_recovery_required',
(session or {}).get('data_recovery_required'))}
def _reconcile(path, root, handles, stack, *, release=True):
@@ -388,8 +397,11 @@ def handoff_session(path, *, evidence_root, process=None, progress=None, stop=No
return result
finally:
with _observer_guard:
_observers.pop(key, None)
if result and result['status'] == 'RELEASED':
if _observers.get(key, (None,))[0] is threading.current_thread():
_observers.pop(key, None)
if (result and result['status'] == 'RELEASED'
and _retained_libraries.get(key) is aliases
and _retained_results.get(key, {}).get('operation_id') == session.get('operation_id')):
_retained_libraries.pop(key, None)
_retained_results.pop(key, None)
thread = threading.Thread(target=observe, name='retained-native-session', daemon=True)
@@ -420,9 +432,16 @@ def discover_sessions(evidence_root, *, library=None):
if not (name.startswith('observation-') and name.endswith('.json')):
continue
observation = read_session_record(Path(folder)/name, evidence_root=root)
proof = observation.get('release_proof') or {}
proof = proof if isinstance(proof, dict) else {}
proof_scope = proof.get('resource_scope')
session_scope = value.get('resource_scope')
if observation['operation_id'] == value['operation_id'] and (
observation.get('status') == 'RELEASED' and observation.get('release_proof')
observation.get('status') == 'RELEASED'
and proof.get('lease_state') == 'RELEASED'
and isinstance(proof_scope, dict) and proof_scope.get('scope_state') == 'OBSERVED'
or observation.get('status') == 'FINISHED'
and isinstance(session_scope, dict) and session_scope.get('scope_state') == 'DECLARED'
and observation.get('native_release_confirmed') is True
and observation.get('worker_exited') is True):
completed = True
+388
View File
@@ -0,0 +1,388 @@
"""Resource release must not inherit another project's session or late evidence."""
from contextlib import contextmanager
from concurrent.futures import ThreadPoolExecutor
import json
import subprocess
import sys
import threading
import pytest
import win32con
import win32file
from cbb_refresh_verify import library_lock as locks
from cbb_refresh_verify import residual_sessions, session_record
def library_file(tmp_path, name='A'):
path = tmp_path/name/'lib.lmc'
path.parent.mkdir()
path.touch()
return path
def cleanup(lease):
with locks._guard:
owner = locks._owners.pop(lease.key, None)
if owner:
owner['handle'].Close()
def test_residual_release_preserves_selected_scope_and_releases_parent_sidecar(tmp_path, monkeypatch):
library = library_file(tmp_path)
other = library.parent/'ReusableBlocksLP/B/Work/database/cdbsvr/sAddress.adr'
other.parent.mkdir(parents=True)
other.write_bytes(b'other native address')
monkeypatch.setattr('cbb_refresh_verify.owner_probe.query_owners', lambda *_: [])
monkeypatch.setattr(residual_sessions, 'query_process_snapshot', lambda: [])
monkeypatch.setattr(residual_sessions, '_process_windows', lambda _: {})
with locks.LibraryLock(library, block_name='A', safe_to_release=lambda: False) as lease:
pass
script = ('import sys; from cbb_refresh_verify.library_lock import LibraryLock; '
'lease=LibraryLock(sys.argv[1],checkpoint=False); lease.__enter__(); lease.__exit__()')
try:
blocked = subprocess.run([sys.executable, '-c', script, str(library)], capture_output=True)
assert blocked.returncode != 0
result = residual_sessions.release_residual_sessions(
{'library': str(library), 'block_names': ['A'], 'sessions': []}, tmp_path/'evidence')
assert result['status'] == 'RELEASE_CONFIRMED', result.get('detail')
assert result['proof']['proof_block_name'] == 'A'
assert result['proof']['lease_state'] == 'RELEASED'
assert other.read_bytes() == b'other native address'
acquired = subprocess.run([sys.executable, '-c', script, str(library)], capture_output=True)
assert acquired.returncode == 0, acquired.stderr
assert lease.path.is_file()
finally:
cleanup(lease)
def test_unknown_checkpoint_does_not_require_explicit_other_project_to_exit(tmp_path, monkeypatch):
library = library_file(tmp_path)
marker = locks.blocker_path(library)
marker.write_bytes(b'{legacy unrecognized checkpoint')
other = {'pid': 123456, 'name': 'ExpeditionPCB.exe',
'command': f'ExpeditionPCB.exe "{tmp_path}/Other/board.prj"'}
monkeypatch.setattr('cbb_refresh_verify.owner_probe.query_owners', lambda *_: [other])
monkeypatch.setattr('cbb_refresh_verify.native_engineering_identity._process_snapshot', lambda: [])
result = locks.release_retained_library(library, tmp_path/'evidence')
assert result['status'] == 'RELEASE_CONFIRMED'
assert result['unknown_checkpoint_archived'] is True
assert result['data_recovery_confirmed'] is False
def test_slow_release_of_a_does_not_block_b_cached_or_lease_checks(tmp_path, monkeypatch):
a, b = library_file(tmp_path, 'A'), library_file(tmp_path, 'B')
entered, finish = threading.Event(), threading.Event()
with locks.LibraryLock(a, safe_to_release=lambda: False) as lease:
pass
@contextmanager
def slow_proof(*args, **kwargs):
entered.set()
assert finish.wait(5)
yield {'exclusive_files': [str(a)]}
monkeypatch.setattr(locks, '_prove_library_released', slow_proof)
def inspect_b():
assert not locks.has_retained_library(b)
assert session_record.library_available(b)
with locks.LibraryLock(b):
return True
with ThreadPoolExecutor(2) as pool:
release = pool.submit(locks.release_retained_library, a, tmp_path/'evidence')
try:
assert entered.wait(2)
check = pool.submit(inspect_b)
assert check.result(timeout=.5)
finally:
finish.set()
release.result(5)
cleanup(lease)
def test_late_observer_cannot_clear_new_operation(tmp_path, monkeypatch):
path = tmp_path/'session.json'
library = tmp_path/'lib.lmc'
session_record.write_session_record(path, {'operation_id': 'old', 'library': str(library)})
key = str(path.absolute()).casefold()
entered, finish = threading.Event(), threading.Event()
def observe(*args, **kwargs):
entered.set()
assert finish.wait(3)
return {'status': 'RELEASED', 'operation_id': 'old'}
monkeypatch.setattr(session_record, 'observe_retained_session', observe)
monkeypatch.setattr(session_record, '_observers', {})
monkeypatch.setattr(session_record, '_retained_libraries', {})
monkeypatch.setattr(session_record, '_retained_results', {})
session_record.handoff_session(path, evidence_root=tmp_path)
assert entered.wait(2)
with session_record._observer_guard:
thread = session_record._observers[key][0]
session_record._retained_libraries[key] = {session_record._library_key(library)}
session_record._retained_results[key] = {'operation_id': 'new', 'status': 'ACTIVE'}
finish.set()
thread.join(3)
assert not session_record.library_available(library)
assert session_record.retained_session_states(library)[0]['operation_id'] == 'new'
@pytest.mark.parametrize('repair_required', [False, True])
def test_target_address_uses_native_inspection_and_keeps_recovery_separate(tmp_path, monkeypatch, repair_required):
library = library_file(tmp_path)
work = library.parent/'ReusableBlocksLP/A/Work'
database = work/'database'
address = database/'cdbsvr/sAddress.adr'
address.parent.mkdir(parents=True)
address.write_bytes(b'native address retained')
(database/'icdb.dat').write_bytes(b'database')
project = work/'A.prj'
project.write_text('KEY iCDBDir "database"\n')
monkeypatch.setattr('cbb_refresh_verify.owner_probe.query_owners', lambda *_: [])
def inspect(target, root, **kwargs):
assert kwargs['inspection_only'] is True
output = f'Loading project file [{project}].\nChecking if project is in use.\n'
output += "Project is not in use. Full iCDB Project Backup's functionality is available.\n"
if repair_required:
output += (f'Project database [{database}] is inconsistent. It has been manually copied while '
"the iCDB Server was running. Please use iCDB Project Backup 'Repair project' functionality to fix this issue.\n")
output += 'Backup information loaded correctly.'
return dict(status='NOT_IN_USE', repair_required=repair_required, inspection_only=True,
repair_requested=False, native_pid=12345, native_exit_code=0,
release={'native_exited': True}, project=str(project), database=str(database), output=output)
monkeypatch.setattr('cbb_refresh_verify.native_repair.run_native_repair', inspect)
with locks.LibraryLock(library, block_name='A', safe_to_release=lambda: False) as lease:
pass
try:
proof = locks.release_retained_library(library, tmp_path/'evidence', block_name='A')
assert proof['lease_state'] == 'RELEASED'
assert proof['data_recovery_required'] is repair_required
assert proof['data_recovery_confirmed'] is False
assert proof['resource_scope']['databases'] == [str(database)]
assert address.read_bytes() == b'native address retained'
assert not locks.has_retained_library(library)
finally:
cleanup(lease)
def test_unknown_checkpoint_excludes_explicit_other_export_server(tmp_path, monkeypatch):
library = library_file(tmp_path)
locks.blocker_path(library).write_bytes(b'unknown')
other = {'pid': 123456, 'name': 'icdbnetserver.exe',
'command': f'icdbnetserver.exe -icdb "{tmp_path}/Other/database"'}
monkeypatch.setattr('cbb_refresh_verify.owner_probe.query_owners', lambda *_: [other])
monkeypatch.setattr('cbb_refresh_verify.native_engineering_identity._process_snapshot', lambda: [other])
proof = locks.release_retained_library(library, tmp_path/'evidence')
assert proof['status'] == 'RELEASE_CONFIRMED'
def test_file_proof_holds_real_handle_until_context_exit(tmp_path):
from cbb_refresh_verify import file_probe
library = library_file(tmp_path)
with file_probe.hold_release_files(library) as proof:
assert proof['exclusive_files'] == [str(library)]
with pytest.raises(Exception):
handle = win32file.CreateFile(str(library), win32con.GENERIC_WRITE, 0, None,
win32con.OPEN_EXISTING, 0, None)
handle.Close()
handle = win32file.CreateFile(str(library), win32con.GENERIC_WRITE, 0, None,
win32con.OPEN_EXISTING, 0, None)
handle.Close()
@pytest.mark.parametrize('cancel', [False, True])
def test_file_probe_timeout_or_stop_reaps_only_its_helper(tmp_path, monkeypatch, cancel):
import time
from cbb_refresh_verify import file_probe
library = library_file(tmp_path)
actual_popen = subprocess.Popen
helpers = []
def blocked_probe(command, **kwargs):
child = actual_popen([sys.executable, '-c', 'import time; time.sleep(30)'], **kwargs)
helpers.append(child)
return child
monkeypatch.setattr(file_probe.subprocess, 'Popen', blocked_probe)
stop = threading.Event()
if cancel:
threading.Timer(.15, stop.set).start()
started = time.monotonic()
with pytest.raises((TimeoutError, InterruptedError)):
with file_probe.hold_release_files(library, timeout_seconds=.3, stop=stop):
pytest.fail('a stalled helper supplied release proof')
assert time.monotonic()-started < 3
assert helpers and helpers[0].poll() is not None
def test_file_proof_survives_probe_exit_during_parent_commit(tmp_path, monkeypatch):
from cbb_refresh_verify import file_probe
library = library_file(tmp_path)
launch = subprocess.Popen
helpers = []
def capture(*args, **kwargs):
child = launch(*args, **kwargs)
helpers.append(child)
return child
monkeypatch.setattr(file_probe.subprocess, 'Popen', capture)
with file_probe.hold_release_files(library):
helpers[0].terminate()
helpers[0].wait(3)
with pytest.raises(Exception):
handle = win32file.CreateFile(str(library), win32con.GENERIC_WRITE, 0, None,
win32con.OPEN_EXISTING, 0, None)
handle.Close()
def test_worker_journal_records_declared_resource_scope_and_operation_identity(tmp_path):
import os
from types import SimpleNamespace
library = library_file(tmp_path)
project = library.parent/'ReusableBlocksLP/A/Work/A.prj'
pcb = project.parent/'PCB/A.pcb'
worker = SimpleNamespace(pid=os.getpid())
path = tmp_path/'session.json'
session_record.register_worker_session(worker, dict(session_record_path=str(path),
operation_id='operation-123', operation='verify', library=str(library), block='A',
project=str(project), pcb=str(pcb)))
record = session_record.read_session_record(path, evidence_root=tmp_path)
assert record['lease_state'] == 'ACTIVE'
assert record['resource_scope']['operation_id'] == 'operation-123'
assert record['resource_scope']['projects'] == [str(project)]
assert str(pcb.parent/'database') in record['resource_scope']['databases']
assert record['data_recovery_required'] is None
def test_replacement_lease_cannot_be_cleared_by_earlier_probe(tmp_path, monkeypatch):
library = library_file(tmp_path)
with locks.LibraryLock(library, safe_to_release=lambda: False) as lease:
pass
marker = locks.blocker_path(library)
original = marker.read_bytes()
@contextmanager
def replaced(*args, **kwargs):
with locks._guard:
locks._owners[lease.key] = {**locks._owners[lease.key], 'lease_id': 'b'*32}
yield {'exclusive_files': [str(library)]}
monkeypatch.setattr(locks, '_prove_library_released', replaced)
try:
with pytest.raises(locks.LibraryBusy):
locks.release_retained_library(library, tmp_path/'evidence')
assert marker.read_bytes() == original
assert locks._owners[lease.key]['lease_id'] == 'b'*32
finally:
cleanup(lease)
def test_multiple_selected_projects_do_not_expand_release_to_unselected_c(tmp_path, monkeypatch):
library = library_file(tmp_path)
other = library.parent/'ReusableBlocksLP/C/Work/database/cdbsvr/sAddress.adr'
other.parent.mkdir(parents=True)
other.write_bytes(b'not selected')
monkeypatch.setattr('cbb_refresh_verify.owner_probe.query_owners', lambda *_: [])
monkeypatch.setattr(residual_sessions, 'query_process_snapshot', lambda: [])
monkeypatch.setattr(residual_sessions, '_process_windows', lambda _: {})
with locks.LibraryLock(library, block_name='A', safe_to_release=lambda: False) as lease:
pass
try:
result = residual_sessions.release_residual_sessions(
{'library': str(library), 'block_names': ['A', 'B'], 'sessions': []}, tmp_path/'evidence')
assert result['status'] == 'RELEASE_CONFIRMED', result.get('detail')
assert result['proof']['proof_block_names'] == ['A', 'B']
assert other.read_bytes() == b'not selected'
finally:
cleanup(lease)
@pytest.mark.parametrize('verdict', ['IN_USE', 'UNCONFIRMED', 'NOT_EXITED'])
def test_native_inspection_without_release_keeps_lease(tmp_path, monkeypatch, verdict):
library = library_file(tmp_path)
work = library.parent/'ReusableBlocksLP/A/Work'
database = work/'database'
address = database/'cdbsvr/sAddress.adr'
address.parent.mkdir(parents=True)
address.write_bytes(b'keep native address')
(database/'icdb.dat').touch()
project = work/'A.prj'
project.write_text('KEY iCDBDir "database"\n')
monkeypatch.setattr('cbb_refresh_verify.owner_probe.query_owners', lambda *_: [])
output = f'Loading project file [{project}].\nChecking if project is in use.\nProject is in use.'
monkeypatch.setattr('cbb_refresh_verify.native_repair.run_native_repair', lambda *a, **kw:
dict(status=verdict, project=str(project), database=str(database), output=output,
inspection_only=True, repair_requested=False, repair_required=False, native_pid=123,
native_exit_code=0, release={'native_exited': verdict != 'NOT_EXITED'}))
with locks.LibraryLock(library, block_name='A', safe_to_release=lambda: False) as lease:
pass
try:
with pytest.raises(locks.LibraryBusy):
locks.release_retained_library(library, tmp_path/'evidence', block_name='A')
assert locks.has_retained_library(library)
assert address.read_bytes() == b'keep native address'
finally:
cleanup(lease)
@pytest.mark.parametrize('invalid_scope', [False, True])
def test_legacy_release_receipt_without_resource_scope_is_rechecked(tmp_path, invalid_scope):
library = library_file(tmp_path)
session_record.write_session_record(tmp_path/'session.json',
{'operation_id': 'legacy', 'library': str(library), 'processes': []})
session_record.write_session_record(tmp_path/'observation-release-old.json',
{'operation_id': 'legacy', 'status': 'RELEASED',
'release_proof': {'status': 'RELEASE_CONFIRMED', 'library': str(library),
**({'lease_state': 'RELEASED', 'resource_scope': None} if invalid_scope else {})}})
found = session_record.discover_sessions(tmp_path)
assert len(found) == 1
assert found[0]['status'] == 'UNCONFIRMED'
assert found[0]['resource_scope']['scope_state'] == 'RECHECK_REQUIRED'
def test_stalled_transaction_checkpoint_does_not_block_other_library_memory(tmp_path, monkeypatch):
from cbb_refresh_verify import management_transaction
a, b = library_file(tmp_path, 'A'), library_file(tmp_path, 'B')
entered, finish = threading.Event(), threading.Event()
original = locks.checkpoint_record
def checkpoint(path):
if path == locks.blocker_path(a) and threading.current_thread().name.startswith('ThreadPool'):
entered.set()
assert finish.wait(5)
return original(path)
with locks.LibraryLock(a):
monkeypatch.setattr(locks, 'checkpoint_record', checkpoint)
with ThreadPoolExecutor(2) as pool:
transaction = pool.submit(management_transaction.current_lease, a)
try:
assert entered.wait(2)
check = pool.submit(locks.has_retained_library, b)
assert check.result(.5) is False
finally:
finish.set()
with pytest.raises(ValueError): # This reader is not the lease's owning thread.
transaction.result(3)
def test_file_proof_holds_address_and_project_binding_against_changes(tmp_path):
from cbb_refresh_verify import file_probe
library = library_file(tmp_path)
database = library.parent/'ReusableBlocksLP/A/Work/database'
address = database/'cdbsvr/sAddress.adr'
address.parent.mkdir(parents=True)
address.write_bytes(b'address')
(database/'icdb.dat').touch()
project = database.parent/'A.prj'
project.write_text('KEY iCDBDir "database"\n')
with file_probe.hold_release_files(library):
for path in (address, project):
with pytest.raises(OSError):
path.write_bytes(b'changed binding')
assert address.read_bytes() == b'address'
assert project.read_text() == 'KEY iCDBDir "database"\n'
def test_file_probe_rejects_scope_junction_even_inside_same_library(tmp_path):
import _winapi
from cbb_refresh_verify import file_probe
library = library_file(tmp_path)
other = library.parent/'ReusableBlocksLP/B/Work'
other.mkdir(parents=True)
selected = library.parent/'ReusableBlocksLP/A/Work'
selected.parent.mkdir(parents=True)
_winapi.CreateJunction(str(other), str(selected))
with pytest.raises(OSError):
with file_probe.hold_release_files(library, (selected,)):
pytest.fail('scope link silently changed the release target')