fix: bind release receipts and external export ownership
This commit is contained in:
1 parent
b070837acb
commit
0440b2efbf
6 files changed
+262
-12
No files matched your search
@@ -96,3 +96,19 @@ Actual summary: `409 passed in 86.98s (0:01:26)`; exit code 0. `git diff --check
|
||||
- Native inspection runs through the existing independent inspection-only supervisor and is never killed by the pure file-probe timeout. Its diagnostics use retained local temporary directories; complete native result records are also embedded in release evidence. Native tool behavior, native close timing, and on-site reproduction remain unverified here.
|
||||
- Ambiguous/shared addresses without an exact unique PRJ-to-database mapping remain protected for manual investigation. No address age, unreachable endpoint, process absence alone, or metadata PID is accepted as ownership/release authority.
|
||||
- No automatic Repair or automatic whole-library backup was added. Full application cancellation/batch/shutdown policy and UI use of the added fields belong to subsequent tasks. Frozen packaging was covered by existing packaging tests and static helper dispatch import, not a new EXE build.
|
||||
|
||||
## Review fix round 1
|
||||
|
||||
Read the independent review in full; both P1 findings were reproduced before source changes. Fixes are ready for scoped re-review; this report does not claim review acceptance. No public signatures changed and the data-damage-not-occupancy ruling remains unchanged.
|
||||
|
||||
1. **Scoped durable release receipts:** added `_proof_covers_session` at the common receipt writer and the durable discovery trust point. A completed/observed proof must match the exact library; a BLOCK/BLOCKS proof must cover the journal's block and have matching observed block membership. The existing operation-ID comparison still prevents a replaced operation receiving an old receipt. Full-library proof remains supported. Single- and multi-block regressions prove an unselected B journal receives no receipt and remains discoverable after clearing memory; additional discovery regressions reject erroneous receipts created by the prior implementation. This fixes the backend sink, rather than filtering UI rows.
|
||||
2. **Unknown checkpoint external exports:** outside-library command paths alone no longer exclude an exporter/server. The independent `export-owner.json` must identify the exact copy root/project and a valid owner library/lease/token, and export process evidence must match the current full process snapshot's PID, creation time and executable. Missing association, missing identity, or stale identity stays unknown and blocks release; an associated copy remains protected. A proven different-library copy is allowed. Tests obtain owner records through the real `begin_export_analysis`, simulate interpreter restart, and cover both exporter and server descendant. Their PRJ CentralLibrary deliberately points to a different library than the export owner, proving that PRJ reference is not used as ownership authority. The old arbitrary `Other/database` fixture was corrected to expect unknown/protected, not treated as independent ownership evidence.
|
||||
|
||||
Red/green commands (the same absolute-import runner is used throughout):
|
||||
|
||||
- `python -X utf8 .superpowers/sdd/2026-11-16-issue15-runtime-unblock/run_tests.py -q tests/test_issue15_release_receipts.py` → **4 failed, 1 passed in 1.82 s** before the fix. Subsequent receipt/session/review covering run → **49 passed in 7.92 s**.
|
||||
- `python -X utf8 .superpowers/sdd/2026-11-16-issue15-runtime-unblock/run_tests.py -q tests/test_issue15_unknown_exports.py tests/test_issue15_resource_release.py -k export` → **5 failed, 2 passed, 21 deselected in 3.93 s** before the fix; → **7 passed, 21 deselected in 3.63 s** after the fix.
|
||||
- Final covering run repeats the 22 original modules plus `test_issue15_release_receipts.py` and `test_issue15_unknown_exports.py`, with `--junitxml=.superpowers/sdd/2026-11-16-issue15-runtime-unblock/task-1-round1-tests.xml`. Exact command and output are in `task-1-round1-tests.log`: **420 passed in 94.43 s (0:01:34)**, exit code 0, no skipped tests.
|
||||
- `python -X utf8 -m compileall -q src/cbb_refresh_verify/session_record.py src/cbb_refresh_verify/native_export_guard.py tests/test_issue15_release_receipts.py tests/test_issue15_unknown_exports.py tests/test_issue15_resource_release.py` and `git diff --check` passed.
|
||||
|
||||
Process evidence is mocked only at the native snapshot boundary; tests do not launch Mentor. Unknown-copy association uses `_process_snapshot` (full `created`/image evidence), not the abbreviated `query_owners` rows. Incomplete runtime identity therefore remains unknown. The previously documented synchronous checkpoint/export-evidence deadline/handoff requirement remains assigned to Task 2/4. No lifecycle, UI, queue, or native worker expansion was included in this fix round.
|
||||
@@ -100,6 +100,77 @@ def begin_export_analysis(library, root, project):
|
||||
return record['token']
|
||||
|
||||
|
||||
def _external_export_library(process):
|
||||
"""Recover a live copy's owner; an outside path alone proves no association."""
|
||||
import ntpath
|
||||
import re
|
||||
import stat
|
||||
import subprocess
|
||||
from .failed_session import _command_args
|
||||
from .backup_recovery import _native_scope_match
|
||||
from .native_engineering_identity import _process_birth
|
||||
def read(path):
|
||||
info = path.lstat()
|
||||
if info.st_file_attributes & stat.FILE_ATTRIBUTE_REPARSE_POINT or info.st_size > 1024*1024:
|
||||
raise ValueError('Untrusted export association')
|
||||
value = json.loads(path.read_text('utf-8'))
|
||||
if not isinstance(value, dict):
|
||||
raise ValueError('Invalid export association')
|
||||
return value
|
||||
try:
|
||||
birth = _process_birth(process.get('created'))
|
||||
image = process.get('executable')
|
||||
if birth is None or not image or ntpath.basename(image).casefold() != process['name'].casefold():
|
||||
return None
|
||||
candidates = []
|
||||
for arg in _command_args(process.get('command', ''))[1:]:
|
||||
path = Path(arg.split('=', 1)[-1])
|
||||
if path.is_absolute() and (path.suffix.casefold() in ('.prj', '.pcb')
|
||||
or path.name.casefold() in ('database', 'icdb.dat')):
|
||||
candidates.append(path)
|
||||
associations = set()
|
||||
for path in candidates:
|
||||
project_root = next((p for p in (path, *path.parents) if p.name.casefold() == 'project'), None)
|
||||
if project_root is None or project_root.resolve() != project_root:
|
||||
return None
|
||||
root = project_root.parent
|
||||
owner = read(root/'export-owner.json')
|
||||
project, library = Path(owner['project']), Path(owner['library'])
|
||||
if (owner.get('protocol') != 'cbb-native-export-analysis' or owner.get('version') != 1
|
||||
or not re.fullmatch('[0-9a-f]{32}', str(owner.get('token', '')))
|
||||
or not re.fullmatch('[0-9a-f]{32}', str(owner.get('lease_id', '')))
|
||||
or type(owner.get('supervisor_pid')) is not int or owner['supervisor_pid'] <= 0
|
||||
or Path(owner['analysis_root']) != root or project.parent != project_root
|
||||
or project.suffix.casefold() != '.prj' or not project.is_file()
|
||||
or not library.is_absolute() or library.suffix.casefold() != '.lmc' or not library.is_file()
|
||||
or root.is_relative_to(library.parent) or library.parent.is_relative_to(root)
|
||||
or _native_scope_match(process['command'], roots=(project_root,)) is not True):
|
||||
return None
|
||||
matched = False
|
||||
for kind, executable in (('ascii', 'icdb2ascii.exe'), ('csv', 'icdb2csv.exe'), ('edif', 'edifexporter.exe')):
|
||||
evidence = root/(kind+'-process.json')
|
||||
if not evidence.exists():
|
||||
continue
|
||||
recorded = read(evidence)
|
||||
command = recorded.get('command')
|
||||
if (recorded.get('native_started') is not True or recorded.get('kind') != kind
|
||||
or not isinstance(command, list) or not command or not all(isinstance(v, str) for v in command)
|
||||
or ntpath.basename(command[0]).casefold() != executable
|
||||
or _native_scope_match(subprocess.list2cmdline(command), roots=(project_root,)) is not True):
|
||||
continue
|
||||
for identity in recorded.get('process_identities', []):
|
||||
if (identity.get('pid') == process['pid'] and _process_birth(identity.get('created')) == birth
|
||||
and identity.get('executable')
|
||||
and ntpath.normcase(identity['executable']) == ntpath.normcase(image)):
|
||||
matched = True
|
||||
if not matched:
|
||||
return None
|
||||
associations.add(library.resolve())
|
||||
return next(iter(associations)) if len(associations) == 1 else None
|
||||
except (OSError, ValueError, KeyError, TypeError, AttributeError):
|
||||
return None
|
||||
|
||||
|
||||
@contextmanager
|
||||
def prove_export_copies_released(library, *, allow_unknown_checkpoint=False, unstarted_token=''):
|
||||
"""Fresh process and exclusive-copy proof, including after interpreter exit."""
|
||||
@@ -116,11 +187,12 @@ def prove_export_copies_released(library, *, allow_unknown_checkpoint=False, uns
|
||||
raise LibraryBusy('Invalid library checkpoint cannot release export ownership.')
|
||||
# Only the explicit release path supplies full-library/native proof and
|
||||
# archives the unchanged metadata while its coordination handle is held.
|
||||
from .library_lock import _owner_matches_scope
|
||||
if any(p['name'].casefold() in _OWNER_NAMES and
|
||||
_owner_matches_scope(p, library, (library.parent,)) is not False
|
||||
for p in _process_snapshot()):
|
||||
raise LibraryBusy('Unknown export checkpoint still has a possible native identity owner.')
|
||||
for process in _process_snapshot():
|
||||
if process['name'].casefold() not in _OWNER_NAMES:
|
||||
continue
|
||||
associated_library = _external_export_library(process)
|
||||
if associated_library is None or associated_library == library:
|
||||
raise LibraryBusy('Unknown export checkpoint still has a possible native identity owner.')
|
||||
yield []
|
||||
return
|
||||
if checkpoint is None or 'export_analysis' not in checkpoint:
|
||||
|
||||
@@ -67,6 +67,25 @@ def acknowledge_library_release(proof, snapshot, *, block_name=None):
|
||||
_retained_results.pop(path, None)
|
||||
|
||||
|
||||
def _proof_covers_session(proof, session):
|
||||
"""A receipt may acknowledge only resources actually covered by its proof."""
|
||||
scope = proof.get('resource_scope')
|
||||
library = session.get('library')
|
||||
if (proof.get('status') != 'RELEASE_CONFIRMED' or proof.get('lease_state') != 'RELEASED'
|
||||
or not isinstance(scope, dict) or scope.get('scope_state') != 'OBSERVED'
|
||||
or not library or _library_key(proof.get('library', '')) != _library_key(library)
|
||||
or _library_key(scope.get('library', '')) != _library_key(library)):
|
||||
return False
|
||||
if proof.get('proof_scope') == 'LIBRARY':
|
||||
return not scope.get('block_names') and not proof.get('proof_block_names')
|
||||
blocks = proof.get('proof_block_names')
|
||||
observed = scope.get('block_names')
|
||||
if (proof.get('proof_scope') not in ('BLOCK', 'BLOCKS') or not isinstance(blocks, list)
|
||||
or not blocks or not isinstance(observed, list) or blocks != observed):
|
||||
return False
|
||||
return bool(session.get('block') and session['block'] in blocks)
|
||||
|
||||
|
||||
def record_session_release(proof, records):
|
||||
"""Persist a past proof only for the pre-proof operation, never replay it."""
|
||||
errors = []
|
||||
@@ -78,7 +97,8 @@ def record_session_release(proof, records):
|
||||
path = Path(record['path'])
|
||||
session = read_session_record(path, evidence_root=path.parent)
|
||||
if (session['operation_id'] != record.get('operation_id')
|
||||
or Path(session['library']).resolve() != library):
|
||||
or Path(session['library']).resolve() != library
|
||||
or not _proof_covers_session(proof, session)):
|
||||
continue
|
||||
write_session_record(path.with_name('observation-release-'+uuid4().hex+'.json'), {
|
||||
'operation_id': session['operation_id'], 'library': str(library),
|
||||
@@ -434,12 +454,10 @@ def discover_sessions(evidence_root, *, library=None):
|
||||
observation = read_session_record(Path(folder)/name, evidence_root=root)
|
||||
proof = observation.get('release_proof') or {}
|
||||
proof = proof if isinstance(proof, dict) else {}
|
||||
proof_scope = proof.get('resource_scope')
|
||||
session_scope = value.get('resource_scope')
|
||||
if observation['operation_id'] == value['operation_id'] and (
|
||||
observation.get('status') == 'RELEASED'
|
||||
and proof.get('lease_state') == 'RELEASED'
|
||||
and isinstance(proof_scope, dict) and proof_scope.get('scope_state') == 'OBSERVED'
|
||||
and _proof_covers_session(proof, value)
|
||||
or observation.get('status') == 'FINISHED'
|
||||
and isinstance(session_scope, dict) and session_scope.get('scope_state') == 'DECLARED'
|
||||
and observation.get('native_release_confirmed') is True
|
||||
|
||||
@@ -0,0 +1,72 @@
|
||||
"""A durable receipt cannot expand the scope of its resource proof."""
|
||||
import pytest
|
||||
|
||||
from cbb_refresh_verify import library_lock as locks, residual_sessions, session_record as sessions
|
||||
|
||||
|
||||
@pytest.mark.parametrize('selected', [('A',), ('A', 'C')])
|
||||
def test_scoped_release_receipt_leaves_unselected_journal_discoverable(tmp_path, monkeypatch, selected):
|
||||
library = tmp_path/'library/lib.lmc'
|
||||
library.parent.mkdir()
|
||||
library.touch()
|
||||
address = library.parent/'ReusableBlocksLP/B/Work/database/cdbsvr/sAddress.adr'
|
||||
address.parent.mkdir(parents=True)
|
||||
address.write_bytes(b'unselected B owner')
|
||||
history = tmp_path/'history'
|
||||
records = []
|
||||
for block in ('A', 'B', 'C'):
|
||||
path = history/block/'session.json'
|
||||
path.parent.mkdir(parents=True)
|
||||
value = dict(operation_id='op-'+block, library=str(library), block=block, processes=[])
|
||||
sessions.write_session_record(path, value)
|
||||
records.append({**value, 'path': str(path)})
|
||||
monkeypatch.setattr('cbb_refresh_verify.owner_probe.query_owners', lambda *_: [])
|
||||
monkeypatch.setattr(residual_sessions, 'query_process_snapshot', lambda: [])
|
||||
monkeypatch.setattr(residual_sessions, '_process_windows', lambda _: {})
|
||||
with locks.LibraryLock(library, block_name='A', safe_to_release=lambda: False) as lease:
|
||||
pass
|
||||
try:
|
||||
result = residual_sessions.release_residual_sessions(dict(library=str(library),
|
||||
block_names=list(selected), sessions=[], runtime_sessions=records), tmp_path/'evidence')
|
||||
assert result['status'] == 'RELEASE_CONFIRMED', result.get('detail')
|
||||
assert not list((history/'B').glob('observation-release-*.json'))
|
||||
assert address.read_bytes() == b'unselected B owner'
|
||||
monkeypatch.setattr(sessions, '_retained_libraries', {})
|
||||
monkeypatch.setattr(sessions, '_retained_results', {})
|
||||
found = sessions.discover_sessions(history, library=library)
|
||||
assert {row['block'] for row in found} == {'A', 'B', 'C'}-set(selected)
|
||||
finally:
|
||||
owner = locks._owners.pop(lease.key, None)
|
||||
if owner:
|
||||
owner['handle'].Close()
|
||||
|
||||
|
||||
@pytest.mark.parametrize('proof_scope,selected', [('BLOCK', ['A']), ('BLOCKS', ['A', 'C'])])
|
||||
def test_discovery_rejects_existing_receipt_for_uncovered_operation(tmp_path, monkeypatch, proof_scope, selected):
|
||||
library = tmp_path/'lib.lmc'
|
||||
library.touch()
|
||||
value = dict(operation_id='op-B', library=str(library), block='B', processes=[])
|
||||
sessions.write_session_record(tmp_path/'session.json', value)
|
||||
proof = dict(status='RELEASE_CONFIRMED', lease_state='RELEASED', library=str(library),
|
||||
proof_scope=proof_scope, proof_block_names=selected,
|
||||
proof_block_name=selected[0] if len(selected) == 1 else None,
|
||||
resource_scope=dict(scope_state='OBSERVED', library=str(library), block_names=selected))
|
||||
# Reproduce a receipt written by the prior implementation, then restart discovery.
|
||||
sessions.write_session_record(tmp_path/'observation-release-old.json',
|
||||
dict(operation_id='op-B', library=str(library), status='RELEASED', release_proof=proof))
|
||||
monkeypatch.setattr(sessions, '_retained_libraries', {})
|
||||
monkeypatch.setattr(sessions, '_retained_results', {})
|
||||
found = sessions.discover_sessions(tmp_path, library=library)
|
||||
assert len(found) == 1 and found[0]['operation_id'] == 'op-B'
|
||||
|
||||
|
||||
def test_receipt_writer_keeps_replaced_operation_unresolved(tmp_path):
|
||||
library = tmp_path/'lib.lmc'
|
||||
library.touch()
|
||||
path = tmp_path/'session.json'
|
||||
sessions.write_session_record(path, dict(operation_id='new', library=str(library), block='A', processes=[]))
|
||||
proof = dict(status='RELEASE_CONFIRMED', lease_state='RELEASED', library=str(library),
|
||||
proof_scope='BLOCK', proof_block_names=['A'], proof_block_name='A',
|
||||
resource_scope=dict(scope_state='OBSERVED', library=str(library), block_names=['A']))
|
||||
sessions.record_session_release(proof, [{'path': str(path), 'operation_id': 'old'}])
|
||||
assert not list(tmp_path.glob('observation-release-*.json'))
|
||||
@@ -162,15 +162,15 @@ def test_target_address_uses_native_inspection_and_keeps_recovery_separate(tmp_p
|
||||
cleanup(lease)
|
||||
|
||||
|
||||
def test_unknown_checkpoint_excludes_explicit_other_export_server(tmp_path, monkeypatch):
|
||||
def test_unknown_checkpoint_does_not_disown_export_server_from_path_alone(tmp_path, monkeypatch):
|
||||
library = library_file(tmp_path)
|
||||
locks.blocker_path(library).write_bytes(b'unknown')
|
||||
other = {'pid': 123456, 'name': 'icdbnetserver.exe',
|
||||
'command': f'icdbnetserver.exe -icdb "{tmp_path}/Other/database"'}
|
||||
monkeypatch.setattr('cbb_refresh_verify.owner_probe.query_owners', lambda *_: [other])
|
||||
monkeypatch.setattr('cbb_refresh_verify.native_engineering_identity._process_snapshot', lambda: [other])
|
||||
proof = locks.release_retained_library(library, tmp_path/'evidence')
|
||||
assert proof['status'] == 'RELEASE_CONFIRMED'
|
||||
with pytest.raises(locks.LibraryBusy):
|
||||
locks.release_retained_library(library, tmp_path/'evidence')
|
||||
|
||||
|
||||
def test_file_proof_holds_real_handle_until_context_exit(tmp_path):
|
||||
|
||||
@@ -0,0 +1,72 @@
|
||||
"""A corrupt checkpoint cannot disown its independent identity-export copy."""
|
||||
import json
|
||||
|
||||
import pytest
|
||||
|
||||
from cbb_refresh_verify import library_lock as locks, native_export_guard as exports
|
||||
|
||||
|
||||
def external_copy(tmp_path, monkeypatch, *, associated, process_name, invalid=None):
|
||||
library = tmp_path/'library/lib.lmc'
|
||||
library.parent.mkdir()
|
||||
library.touch()
|
||||
other = tmp_path/'other/lib.lmc'
|
||||
other.parent.mkdir()
|
||||
other.touch()
|
||||
root = tmp_path/'evidence/identity'
|
||||
(root/'project/database').mkdir(parents=True)
|
||||
project = root/'project/Source.prj'
|
||||
# Source-library references may differ from the transaction's owner library.
|
||||
project.write_text(f'KEY CentralLibrary "{other if associated else library}"\n', 'utf-8')
|
||||
(root/'project/database/icdb.dat').touch()
|
||||
owner_library = library if associated else other
|
||||
with locks.LibraryLock(owner_library) as lease:
|
||||
exports.begin_export_analysis(owner_library, root, project)
|
||||
# Restart removes process-local retained ownership, but independent evidence remains.
|
||||
locks._owners.pop(lease.key)['handle'].Close()
|
||||
locks.blocker_path(library).write_bytes(b'{truncated checkpoint')
|
||||
process = dict(pid=123456, parent=98765, name=process_name,
|
||||
executable='C:/Mentor/'+process_name, created='20261116120000.000000+480',
|
||||
command=(f'icdb2ascii.exe -project="{project}"' if process_name == 'icdb2ascii.exe'
|
||||
else f'icdbnetserver.exe -icdb "{project.parent}/database"'))
|
||||
record = dict(command=['C:/Mentor/icdb2ascii.exe', '-project='+str(project)], kind='ascii',
|
||||
success=False, safe_to_continue=False, native_started=True,
|
||||
pid=process['pid'] if process_name == 'icdb2ascii.exe' else 98765,
|
||||
observed_pids=[process['pid']], root_created='2026-11-16T04:00:00+00:00',
|
||||
process_identities=[{k: process[k] for k in ('pid', 'created', 'executable')}])
|
||||
if invalid == 'identity':
|
||||
record['process_identities'][0]['created'] = '20261115120000.000000+480'
|
||||
(root/'ascii-process.json').write_text(json.dumps(record), 'utf-8')
|
||||
if invalid == 'missing_owner':
|
||||
(root/'export-owner.json').unlink()
|
||||
monkeypatch.setattr('cbb_refresh_verify.native_engineering_identity._retained_exports', [])
|
||||
monkeypatch.setattr('cbb_refresh_verify.owner_probe.query_owners', lambda *_: [process])
|
||||
monkeypatch.setattr('cbb_refresh_verify.native_engineering_identity._process_snapshot', lambda: [process])
|
||||
return library, root
|
||||
|
||||
|
||||
@pytest.mark.parametrize('process_name', ['icdb2ascii.exe', 'icdbnetserver.exe'])
|
||||
@pytest.mark.parametrize('associated', [True, False])
|
||||
def test_corrupt_checkpoint_requires_independent_export_association(tmp_path, monkeypatch, process_name, associated):
|
||||
library, root = external_copy(tmp_path, monkeypatch,
|
||||
associated=associated, process_name=process_name)
|
||||
marker = locks.blocker_path(library)
|
||||
before = marker.read_bytes()
|
||||
if associated:
|
||||
with pytest.raises(locks.LibraryBusy):
|
||||
locks.release_retained_library(library, tmp_path/'release')
|
||||
assert marker.read_bytes() == before
|
||||
else:
|
||||
proof = locks.release_retained_library(library, tmp_path/'release')
|
||||
assert proof['status'] == 'RELEASE_CONFIRMED'
|
||||
assert (root/'export-owner.json').is_file()
|
||||
|
||||
|
||||
@pytest.mark.parametrize('invalid', ['missing_owner', 'identity'])
|
||||
def test_unproven_other_export_remains_unknown(tmp_path, monkeypatch, invalid):
|
||||
library, _ = external_copy(tmp_path, monkeypatch,
|
||||
associated=False, process_name='icdbnetserver.exe', invalid=invalid)
|
||||
before = locks.blocker_path(library).read_bytes()
|
||||
with pytest.raises(locks.LibraryBusy):
|
||||
locks.release_retained_library(library, tmp_path/'release')
|
||||
assert locks.blocker_path(library).read_bytes() == before
|
||||
Reference in new issue
Block a user