"""Привязка документов к версиям консолидации (spec documents-api «Подтверждение документа», «Подхват документов при создании версии и расширении периметра»; audit-trail `CONSOLIDATION_STATUS_CHANGE`; design D-5, D-6, D-9; рулинги R-1 — ключ замены с признаком нарастающего, R-2 — гонки потерянной привязки, R-3 — `FOR NO KEY UPDATE` документа). Сценарии подтверждения и подхвата — поверх `seeded`/`auth_client` (транзакция теста). Гонки — на `committed_*` (настоящие параллельные транзакции, таймаут 5 с): подтверждение ∥ правка периметра в обе стороны, создание версии ∥ подтверждение в обе стороны и совместимость блокировки документа подтверждения со вставкой привязки. """ import asyncio import datetime import uuid from collections.abc import Sequence from typing import Any import pytest from loguru import logger from sqlalchemy import select, text from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker from src.common.clock import utc_now from src.models.access import AppUser from src.models.audit_ex import AuditEvent from src.models.consolidation import ( Consolidation, ConsolidationDocument, ConsolidationMember, ConsolidationStatusHistory, ConsolidationVersion, ) from src.models.directories import CompanyGroup, LegalEntity, RegionalOffice, RelatedGroup from src.models.enums import ( AuditAction, ConsolidationStatus, DocumentStatus, DocumentType, FileStatus, PeriodQuarter, RecognitionStatus, ) from src.models.files import Document, StoredFile from src.repositories import consolidations as repo from src.repositories import files as files_repo from src.services import consolidation_documents from src.services.audit import AuditService from src.services.consolidation_documents import fits from src.services.result_mapping import supersede_previous from tests.conftest import AuthClientFactory from tests.factories import ( make_company_group, make_confirmed_osv, make_consolidation, make_document, make_legal_entity, make_lock, make_membership, make_related_group, make_stored_file, ) HTTP_OK = 200 HTTP_CREATED = 201 ROSTOV = "Ростовский региональный филиал" KA_RF = "ka_rf@armka.local" ACCOUNT = "60" YEAR = 2024 LOCK_WAIT_SECONDS = 5 ROWS = [{"name": "ООО Контрагент", "inn": "7701234567", "closing_debit": 10}] Q1, Q2, Q3, Q4 = PeriodQuarter.Q1, PeriodQuarter.Q2, PeriodQuarter.Q3, PeriodQuarter.Q4 # --- fits: чистая функция (D-5) -------------------------------------------------------------------------------------- _MEMBER = uuid.uuid4() _STRANGER = uuid.uuid4() def _pure_document( *, entity: uuid.UUID = _MEMBER, year: int = YEAR, quarter: PeriodQuarter | None = Q3, cumulative: bool | None ) -> Document: return Document( legal_entity_id=entity, document_type=DocumentType.OSV, osv_account_code=ACCOUNT, period_year=year, period_quarter=quarter, is_cumulative=cumulative, status=DocumentStatus.CONFIRMED, ) @pytest.mark.parametrize( ("document", "expected"), [ (_pure_document(cumulative=False, quarter=Q1), True), # член, квартальный Q1 ≤ Q3 (_pure_document(entity=_STRANGER, cumulative=True), False), # не член периметра (_pure_document(year=YEAR - 1, cumulative=True), False), # другой год (_pure_document(cumulative=True, quarter=Q3), True), # нарастающий за квартал версии (_pure_document(cumulative=True, quarter=Q2), False), # нарастающий за Q2 при версии Q3 (Review Focus 2) (_pure_document(cumulative=False, quarter=Q4), False), # квартальный позже квартала версии (_pure_document(cumulative=False, quarter=None), False), # квартал не определён (_pure_document(cumulative=None, quarter=Q3), True), # ББ/ОФР: NULL — нарастающий (решение №10) (_pure_document(cumulative=None, quarter=Q2), False), # NULL за Q2 при версии Q3 — как нарастающий ], ) def test_fits(document: Document, expected: bool) -> None: version = ConsolidationVersion(status=ConsolidationStatus.INITIATED) consolidation = Consolidation(year=YEAR, quarter=Q3) assert fits(document, version, consolidation, {_MEMBER}) is expected # --- данные --------------------------------------------------------------------------------------------------------- async def _user(session: AsyncSession, email: str = KA_RF) -> AppUser: return (await session.execute(select(AppUser).where(AppUser.email == email))).scalar_one() async def _group(session: AsyncSession, members: int = 2) -> tuple[CompanyGroup, list[LegalEntity]]: office = (await session.execute(select(RegionalOffice).where(RegionalOffice.name == ROSTOV))).scalar_one() group = await make_company_group(session, name=f"ГК {uuid.uuid4().hex[:6]}", office=office) entities = [await make_legal_entity(session, name=f"ООО Член {n}", group=group) for n in range(members)] return group, entities async def _version( session: AsyncSession, group: CompanyGroup, members: Sequence[LegalEntity], *, quarter: PeriodQuarter = Q3, status: ConsolidationStatus = ConsolidationStatus.INITIATED, ) -> ConsolidationVersion: _, version = await make_consolidation( session, group=group, year=YEAR, quarter=quarter, members=members, initiator=await _user(session), status=status ) return version async def _document( # noqa: PLR0913 - тестовая фабрика: ЮЛ, счёт, период и статусы документа и файла session: AsyncSession, entity: LegalEntity, *, quarter: PeriodQuarter = Q2, cumulative: bool = False, account: str = ACCOUNT, status: DocumentStatus = DocumentStatus.NEEDS_CHECK, ) -> Document: """Документ ОСВ файла `PROCESSED` (по умолчанию — квартальный `NEEDS_CHECK` за 2024 Q2).""" file, job = await make_stored_file( session, entity=entity, status=FileStatus.PROCESSED, job_status=RecognitionStatus.done ) return await make_document( session, file=file, job=job, account=account, year=YEAR, quarter=quarter, status=status, is_cumulative=cumulative, ) async def _confirmed( # noqa: PLR0913 - тестовая фабрика: ЮЛ, период, счёт и время подтверждения session: AsyncSession, entity: LegalEntity, *, quarter: PeriodQuarter, cumulative: bool = False, account: str = ACCOUNT, confirmed_at: datetime.datetime | None = None, ) -> Document: document = await make_confirmed_osv( session, entity=entity, account=account, year=YEAR, quarter=quarter, is_cumulative=cumulative, rows=ROWS ) if confirmed_at is not None: document.confirmed_at = confirmed_at await session.flush() return document async def _bound(session: AsyncSession, version_id: uuid.UUID) -> set[uuid.UUID]: stmt = select(ConsolidationDocument.document_id).where(ConsolidationDocument.consolidation_version_id == version_id) return set((await session.execute(stmt)).scalars()) async def _status(session: AsyncSession, version_id: uuid.UUID) -> ConsolidationStatus: stmt = select(ConsolidationVersion.status).where(ConsolidationVersion.id == version_id) return (await session.execute(stmt.execution_options(populate_existing=True))).scalar_one() async def _history(session: AsyncSession, version_id: uuid.UUID) -> list[ConsolidationStatusHistory]: return await repo.history(session, version_id) async def _bind(session: AsyncSession, version: ConsolidationVersion, document: Document) -> None: session.add(ConsolidationDocument(consolidation_version_id=version.id, document_id=document.id)) await session.flush() # --- подтверждение ---------------------------------------------------------------------------------------------------- @pytest.mark.integration async def test_confirm_binds_document_and_moves_version(seeded: AsyncSession, auth_client: AuthClientFactory) -> None: """«Привязка к версии и переход статуса»: квартальная ОСВ 60 за Q2 при версии Q3 в `INITIATED`.""" group, entities = await _group(seeded) version = await _version(seeded, group, entities) document = await _document(seeded, entities[0]) user = await _user(seeded) client = await auth_client(KA_RF) response = await client.post(f"/documents/{document.id}/confirm") assert response.status_code == HTTP_OK, response.text assert await _bound(seeded, version.id) == {document.id} assert await _status(seeded, version.id) is ConsolidationStatus.DATA_CORRECTION last = (await _history(seeded, version.id))[-1] assert (last.from_status, last.to_status, last.changed_by) == ( ConsolidationStatus.INITIATED, ConsolidationStatus.DATA_CORRECTION, user.id, ) [event] = ( await seeded.execute(select(AuditEvent).where(AuditEvent.action == AuditAction.CONSOLIDATION_STATUS_CHANGE)) ).scalars() assert (event.user_id, event.consolidation_version_id, event.object_id) == (user.id, version.id, version.id) assert (event.before_value, event.after_value) == ({"status": "INITIATED"}, {"status": "DATA_CORRECTION"}) @pytest.mark.integration @pytest.mark.parametrize( ("status", "moved"), [ (ConsolidationStatus.DOCUMENTS_PROCESSING, True), (ConsolidationStatus.DATA_CORRECTION, False), # уже там: привязка без перехода ], ) async def test_confirm_binds_in_other_in_work_statuses( seeded: AsyncSession, auth_client: AuthClientFactory, status: ConsolidationStatus, moved: bool ) -> None: group, entities = await _group(seeded) version = await _version(seeded, group, entities, status=status) before = len(await _history(seeded, version.id)) document = await _document(seeded, entities[0], quarter=Q3, cumulative=True) client = await auth_client(KA_RF) response = await client.post(f"/documents/{document.id}/confirm") assert response.status_code == HTTP_OK, response.text assert await _bound(seeded, version.id) == {document.id} assert await _status(seeded, version.id) is ConsolidationStatus.DATA_CORRECTION assert len(await _history(seeded, version.id)) == before + (1 if moved else 0) @pytest.mark.integration async def test_confirm_does_not_bind_cumulative_of_earlier_quarter( seeded: AsyncSession, auth_client: AuthClientFactory ) -> None: """«Не подходит по периоду»: нарастающая ОСВ 60 за Q2 при версии Q3 — привязки нет, статус прежний.""" group, entities = await _group(seeded) version = await _version(seeded, group, entities) document = await _document(seeded, entities[0], quarter=Q2, cumulative=True) client = await auth_client(KA_RF) response = await client.post(f"/documents/{document.id}/confirm") assert response.status_code == HTTP_OK, response.text assert await _bound(seeded, version.id) == set() assert await _status(seeded, version.id) is ConsolidationStatus.INITIATED assert len(await _history(seeded, version.id)) == 1 @pytest.mark.integration async def test_confirm_does_not_bind_cancelled_version(seeded: AsyncSession, auth_client: AuthClientFactory) -> None: """«Версия в CANCELLED»: подходящий документ не привязывается.""" group, entities = await _group(seeded) version = await _version(seeded, group, entities, status=ConsolidationStatus.CANCELLED) document = await _document(seeded, entities[0]) client = await auth_client(KA_RF) response = await client.post(f"/documents/{document.id}/confirm") assert response.status_code == HTTP_OK, response.text assert await _bound(seeded, version.id) == set() assert await _status(seeded, version.id) is ConsolidationStatus.CANCELLED @pytest.mark.integration async def test_confirm_does_not_bind_entity_outside_perimeter( seeded: AsyncSession, auth_client: AuthClientFactory ) -> None: """ЮЛ той же ГК, но вне периметра версии: привязки нет (блокируется надмножество, привязывается только член).""" group, entities = await _group(seeded, members=3) version = await _version(seeded, group, entities[:2]) document = await _document(seeded, entities[2]) client = await auth_client(KA_RF) response = await client.post(f"/documents/{document.id}/confirm") assert response.status_code == HTTP_OK, response.text assert await _bound(seeded, version.id) == set() assert await _status(seeded, version.id) is ConsolidationStatus.INITIATED @pytest.mark.integration async def test_confirm_replaces_bound_document_with_same_key( seeded: AsyncSession, auth_client: AuthClientFactory ) -> None: """Привязанный документ того же ЮЛ с той же тройкой и тем же признаком нарастающего отвязывается (D-5); нарастающий той же пары (ЮЛ, счёт) за тот же квартал остаётся (R-1); документ другого ЮЛ не трогается.""" group, entities = await _group(seeded) version = await _version(seeded, group, entities, status=ConsolidationStatus.DATA_CORRECTION) old_quarterly = await _confirmed(seeded, entities[0], quarter=Q3) cumulative = await _confirmed(seeded, entities[0], quarter=Q3, cumulative=True) other_entity = await _confirmed(seeded, entities[1], quarter=Q3) for document in (old_quarterly, cumulative, other_entity): await _bind(seeded, version, document) new_quarterly = await _document(seeded, entities[0], quarter=Q3) client = await auth_client(KA_RF) response = await client.post(f"/documents/{new_quarterly.id}/confirm") assert response.status_code == HTTP_OK, response.text assert await _bound(seeded, version.id) == {new_quarterly.id, cumulative.id, other_entity.id} @pytest.mark.integration async def test_one_document_binds_to_two_versions(seeded: AsyncSession, auth_client: AuthClientFactory) -> None: """Review Focus 1: консолидации ГК за Q2 и Q3 одного года; квартальная ОСВ Q1 подходит обеим — подтверждение привязывает её к обеим и переводит обе в `DATA_CORRECTION`.""" group, entities = await _group(seeded) q2 = await _version(seeded, group, entities, quarter=Q2) q3 = await _version(seeded, group, entities, quarter=Q3) document = await _document(seeded, entities[0], quarter=Q1) client = await auth_client(KA_RF) response = await client.post(f"/documents/{document.id}/confirm") assert response.status_code == HTTP_OK, response.text for version in (q2, q3): assert await _bound(seeded, version.id) == {document.id} assert await _status(seeded, version.id) is ConsolidationStatus.DATA_CORRECTION events = ( await seeded.execute( select(AuditEvent.consolidation_version_id).where( AuditEvent.action == AuditAction.CONSOLIDATION_STATUS_CHANGE ) ) ).scalars() assert sorted(events) == sorted([q2.id, q3.id]) @pytest.mark.integration async def test_on_document_confirmed_returns_bound_versions(seeded: AsyncSession) -> None: """Хук возвращает id версий, получивших привязку (по возрастанию `id`); неподходящие и аннулированные — нет.""" group, entities = await _group(seeded) q2 = await _version(seeded, group, entities, quarter=Q2) q3 = await _version(seeded, group, entities, quarter=Q3) q1 = await _version(seeded, group, entities, quarter=Q1) # квартальный Q2 не подходит версии Q1 q4 = await _version(seeded, group, entities, quarter=Q4, status=ConsolidationStatus.CANCELLED) document = await _document(seeded, entities[0], quarter=Q2, status=DocumentStatus.CONFIRMED) user = await _user(seeded) bound = await consolidation_documents.on_document_confirmed( seeded, AuditService(seeded), document=document, user_id=user.id, now=utc_now() ) assert bound == sorted([q2.id, q3.id]) assert await _bound(seeded, q1.id) == set() assert await _bound(seeded, q4.id) == set() # --- подхват ---------------------------------------------------------------------------------------------------------- async def _gsk(session: AsyncSession, group: CompanyGroup, entities: Sequence[LegalEntity]) -> RelatedGroup: gsk = await make_related_group(session, group=group, name=f"ГСК {uuid.uuid4().hex[:6]}") for entity in entities: await make_membership(session, related_group=gsk, entity=entity) return gsk def _create_body(group: CompanyGroup, gsk: RelatedGroup, quarter: str = "Q3") -> dict[str, Any]: return {"company_group_id": str(group.id), "related_group_id": str(gsk.id), "year": YEAR, "quarter": quarter} @pytest.mark.integration async def test_create_picks_up_documents_confirmed_before(seeded: AsyncSession, auth_client: AuthClientFactory) -> None: """«Фикстуры, подтверждённые до консолидации»: ОСВ 60 за Q1, Q2, Q3 подтверждены, затем создана консолидация за Q3 — три привязки, версия в `INITIATED` (ячейка чек-листа — Task 8).""" group, entities = await _group(seeded) gsk = await _gsk(seeded, group, entities) documents = [await _confirmed(seeded, entities[0], quarter=quarter) for quarter in (Q1, Q2, Q3)] client = await auth_client(KA_RF) response = await client.post("/consolidations", json=_create_body(group, gsk)) assert response.status_code == HTTP_CREATED, response.text version_id = uuid.UUID(response.json()["id"]) assert await _bound(seeded, version_id) == {document.id for document in documents} assert await _status(seeded, version_id) is ConsolidationStatus.INITIATED assert len(await _history(seeded, version_id)) == 1 async def _latest_with_smaller_id( session: AsyncSession, entity: LegalEntity, *, account: str = ACCOUNT, equal: bool = False ) -> tuple[Document, Document]: """Два `CONFIRMED` документа одного ключа (квартальная ОСВ за Q3): `(победитель, проигравший)`. Без `equal` более поздний `confirmed_at` получает документ с МЕНЬШИМ `id` — выбор «по наибольшему id» вместо «по самому позднему `confirmed_at`» проиграет. С `equal` время одинаковое — побеждает больший `id` (решение №35). """ first = await _confirmed(session, entity, quarter=Q3, account=account) second = await _confirmed(session, entity, quarter=Q3, account=account) smaller, larger = sorted((first, second), key=lambda document: document.id) now = utc_now() smaller.confirmed_at = now larger.confirmed_at = now if equal else now - datetime.timedelta(hours=1) await session.flush() return (larger, smaller) if equal else (smaller, larger) @pytest.mark.integration async def test_create_picks_latest_confirmed_of_same_key(seeded: AsyncSession, auth_client: AuthClientFactory) -> None: """«Две подтверждённые версии одного документа»: два незаменённых файла с `CONFIRMED` ОСВ 62 за Q3 — привязан документ с более поздним `confirmed_at`.""" group, entities = await _group(seeded) gsk = await _gsk(seeded, group, entities) later, _ = await _latest_with_smaller_id(seeded, entities[0], account="62") client = await auth_client(KA_RF) response = await client.post("/consolidations", json=_create_body(group, gsk)) assert response.status_code == HTTP_CREATED, response.text assert await _bound(seeded, uuid.UUID(response.json()["id"])) == {later.id} @pytest.mark.integration async def test_bind_confirmed_filters_documents(seeded: AsyncSession) -> None: """Подхват берёт только `CONFIRMED` документы незаменённых файлов, подходящие версии; квартальный и нарастающий за один квартал — оба (R-1); статус версии не меняется; повторный подхват ничего не добавляет.""" group, entities = await _group(seeded) version = await _version(seeded, group, entities) consolidation = await seeded.get(Consolidation, version.consolidation_id) assert consolidation is not None quarterly = await _confirmed(seeded, entities[0], quarter=Q3) cumulative = await _confirmed(seeded, entities[0], quarter=Q3, cumulative=True) await _confirmed(seeded, entities[0], quarter=Q2, cumulative=True) # нарастающий за Q2 — не подходит await _document(seeded, entities[0], quarter=Q1) # не подтверждён superseded = await _confirmed(seeded, entities[0], quarter=Q1) await seeded.execute( text("UPDATE stored_file SET status = 'SUPERSEDED' WHERE id = :id"), {"id": superseded.stored_file_id} ) count = await consolidation_documents.bind_confirmed( seeded, version=version, consolidation=consolidation, entity_ids=[entities[0].id] ) assert count == 2 # noqa: PLR2004 - квартальный и нарастающий Q3 assert await _bound(seeded, version.id) == {quarterly.id, cumulative.id} assert await _status(seeded, version.id) is ConsolidationStatus.INITIATED again = await consolidation_documents.bind_confirmed( seeded, version=version, consolidation=consolidation, entity_ids=[entities[0].id] ) assert again == 0 assert await _bound(seeded, version.id) == {quarterly.id, cumulative.id} @pytest.mark.integration async def test_bind_confirmed_replaces_older_binding_of_same_key(seeded: AsyncSession) -> None: """Существующая привязка той же тройки заменяется документом с более поздним `confirmed_at`.""" group, entities = await _group(seeded) version = await _version(seeded, group, entities) consolidation = await seeded.get(Consolidation, version.consolidation_id) assert consolidation is not None newer, older = await _latest_with_smaller_id(seeded, entities[0]) await _bind(seeded, version, older) count = await consolidation_documents.bind_confirmed( seeded, version=version, consolidation=consolidation, entity_ids=[entities[0].id] ) assert count == 1 assert await _bound(seeded, version.id) == {newer.id} @pytest.mark.integration async def test_bind_confirmed_tie_on_confirmed_at_takes_larger_id(seeded: AsyncSession) -> None: """Решение №35: при равном `confirmed_at` привязывается документ с большим `id`.""" group, entities = await _group(seeded) version = await _version(seeded, group, entities) consolidation = await seeded.get(Consolidation, version.consolidation_id) assert consolidation is not None winner, _ = await _latest_with_smaller_id(seeded, entities[0], equal=True) count = await consolidation_documents.bind_confirmed( seeded, version=version, consolidation=consolidation, entity_ids=[entities[0].id] ) assert count == 1 assert await _bound(seeded, version.id) == {winner.id} # --- гонки (настоящие параллельные транзакции) ------------------------------------------------------------------------ async def _wait_for_lock_waiter(maker: async_sessionmaker[AsyncSession]) -> None: """Ждёт, пока какой-то сеанс БД встанет в ожидание блокировки (строки или advisory).""" deadline = asyncio.get_running_loop().time() + LOCK_WAIT_SECONDS while asyncio.get_running_loop().time() < deadline: async with maker() as session: waiting = ( await session.execute( text( "SELECT count(*) FROM pg_stat_activity " "WHERE datname = current_database() AND wait_event_type = 'Lock'" ) ) ).scalar_one() if waiting: return await asyncio.sleep(0.05) raise AssertionError("запрос не встал в ожидание блокировки") async def _committed_setup( maker: async_sessionmaker[AsyncSession], *, members: int = 2, spare: int = 1, with_version: bool = True ) -> dict[str, Any]: """ГК филиала: `members` ЮЛ в периметре версии 2024 Q3 (`INITIATED`, блокировка у ka_rf), `spare` ЮЛ ГК вне периметра, ГСК из всех ЮЛ; документ `NEEDS_CHECK` квартальной ОСВ 60 за Q2 у первого запасного ЮЛ (или у первого члена, если запасных нет).""" async with maker() as session, session.begin(): group, entities = await _group(session, members=members + spare) gsk = await _gsk(session, group, entities) version_id = None if with_version: version = await _version(session, group, entities[:members]) await make_lock(session, version=version, user=await _user(session)) version_id = version.id owner = entities[members] if spare else entities[0] document = await _document(session, owner) return { "group_id": group.id, "gsk_id": gsk.id, "entity_ids": [entity.id for entity in entities], "version_id": version_id, "document_id": document.id, "file_id": document.stored_file_id, "owner_id": owner.id, "user_id": (await _user(session)).id, } async def _confirm_in(session: AsyncSession, data: dict[str, Any]) -> list[uuid.UUID]: """Подтверждение, как в `DocumentService.confirm`: файл → документ → статус → хук привязки (транзакция открыта).""" file = await files_repo.lock_file(session, data["file_id"]) document = await files_repo.lock_document(session, data["document_id"]) assert file is not None assert document is not None now = utc_now() document.status = DocumentStatus.CONFIRMED document.confirmed_by = data["user_id"] document.confirmed_at = now await session.flush() return await consolidation_documents.on_document_confirmed( session, AuditService(session), document=document, user_id=data["user_id"], now=now ) async def _committed_bound(maker: async_sessionmaker[AsyncSession], version_id: uuid.UUID) -> set[uuid.UUID]: async with maker() as session: return await _bound(session, version_id) @pytest.mark.integration async def test_confirm_waits_for_member_edit_and_binds_new_member( committed_sessionmaker: async_sessionmaker[AsyncSession], committed_client: AuthClientFactory ) -> None: """R-2: правка периметра (версия `FOR UPDATE`, добавлен новый член) ∥ подтверждение документа нового члена. Подтверждение блокирует надмножество версий ГК, ждёт правку и после её commit видит нового члена — привязка не теряется, версия в `DATA_CORRECTION`.""" maker = committed_sessionmaker data = await _committed_setup(maker) client = await committed_client(KA_RF) async with maker() as editor, editor.begin(): assert await repo.lock_version(editor, data["version_id"]) is not None editor.add(ConsolidationMember(consolidation_version_id=data["version_id"], legal_entity_id=data["owner_id"])) await editor.flush() confirm = asyncio.create_task(client.post(f"/documents/{data['document_id']}/confirm")) await _wait_for_lock_waiter(maker) assert not confirm.done() response = await asyncio.wait_for(confirm, LOCK_WAIT_SECONDS) assert response.status_code == HTTP_OK, response.text assert await _committed_bound(maker, data["version_id"]) == {data["document_id"]} async with maker() as session: assert await _status(session, data["version_id"]) is ConsolidationStatus.DATA_CORRECTION @pytest.mark.integration async def test_member_edit_waits_for_confirm_and_picks_document_up( committed_sessionmaker: async_sessionmaker[AsyncSession], committed_client: AuthClientFactory ) -> None: """R-2/R-3, обратный порядок: подтверждение держит файл, документ и версии ГК; `PUT …/members`, добавляющий ЮЛ документа, ждёт версию, после commit подтверждения подхватывает уже `CONFIRMED` документ. Взаимоблокировки нет.""" maker = committed_sessionmaker data = await _committed_setup(maker) client = await committed_client(KA_RF) body = {"legal_entity_ids": [str(entity_id) for entity_id in data["entity_ids"]]} async with maker() as confirmer, confirmer.begin(): assert await _confirm_in(confirmer, data) == [] # ЮЛ ещё не член — привязывать не к чему edit = asyncio.create_task(client.put(f"/consolidations/versions/{data['version_id']}/members", json=body)) await _wait_for_lock_waiter(maker) assert not edit.done() response = await asyncio.wait_for(edit, LOCK_WAIT_SECONDS) assert response.status_code == HTTP_OK, response.text assert await _committed_bound(maker, data["version_id"]) == {data["document_id"]} async with maker() as session: # подхват статус не меняет assert await _status(session, data["version_id"]) is ConsolidationStatus.INITIATED @pytest.mark.integration async def test_document_lock_does_not_block_binding_insert( committed_sessionmaker: async_sessionmaker[AsyncSession], ) -> None: """R-3: подтверждение держит документ (`lock_document`) и затем ждёт версию; подхват держит версию и вставляет привязку на этот документ (FK берёт `FOR KEY SHARE` документа). С `FOR UPDATE` документа это взаимоблокировка; с `FOR NO KEY UPDATE` вставка проходит, обе транзакции завершаются за таймаут.""" maker = committed_sessionmaker data = await _committed_setup(maker, spare=0) async with maker() as session, session.begin(): document = await session.get(Document, data["document_id"]) assert document is not None document.status = DocumentStatus.CONFIRMED document.confirmed_at = utc_now() version_held = asyncio.Event() async def binder() -> None: async with maker() as session, session.begin(): assert await repo.lock_version(session, data["version_id"]) is not None version_held.set() session.add( ConsolidationDocument(consolidation_version_id=data["version_id"], document_id=data["document_id"]) ) await session.flush() # FK: FOR KEY SHARE документа, который держит «подтверждение» async with maker() as confirmer, confirmer.begin(): assert await files_repo.lock_file(confirmer, data["file_id"]) is not None assert await files_repo.lock_document(confirmer, data["document_id"]) is not None binding = asyncio.create_task(binder()) await asyncio.wait_for(version_held.wait(), LOCK_WAIT_SECONDS) # «подтверждение» ждёт версию, которую держит подхват; с FOR UPDATE документа — взаимоблокировка assert await asyncio.wait_for(repo.lock_version(confirmer, data["version_id"]), LOCK_WAIT_SECONDS) await asyncio.wait_for(binding, LOCK_WAIT_SECONDS) assert await _committed_bound(maker, data["version_id"]) == {data["document_id"]} @pytest.mark.integration async def test_create_waits_for_confirm_and_picks_document_up( committed_sessionmaker: async_sessionmaker[AsyncSession], committed_client: AuthClientFactory ) -> None: """R-2, создание версии ∥ подтверждение: подтверждение держит advisory-блокировку ГК; `POST /consolidations` ждёт её и после commit подхватывает уже `CONFIRMED` документ.""" maker = committed_sessionmaker data = await _committed_setup(maker, spare=0, with_version=False) client = await committed_client(KA_RF) body = {"company_group_id": str(data["group_id"]), "related_group_id": str(data["gsk_id"]), "year": YEAR} async with maker() as confirmer, confirmer.begin(): assert await _confirm_in(confirmer, data) == [] create = asyncio.create_task(client.post("/consolidations", json={**body, "quarter": "Q3"})) await _wait_for_lock_waiter(maker) assert not create.done() response = await asyncio.wait_for(create, LOCK_WAIT_SECONDS) assert response.status_code == HTTP_CREATED, response.text assert await _committed_bound(maker, uuid.UUID(response.json()["id"])) == {data["document_id"]} @pytest.mark.integration async def test_confirm_waits_for_create_and_binds_new_version( committed_sessionmaker: async_sessionmaker[AsyncSession], committed_client: AuthClientFactory ) -> None: """R-2, подтверждение ∥ создание версии: создание держит advisory-блокировку ГК (новая версия до commit невидима); подтверждение ждёт её и после commit привязывает документ к новой версии и двигает её статус.""" maker = committed_sessionmaker data = await _committed_setup(maker, spare=0, with_version=False) client = await committed_client(KA_RF) async with maker() as creator, creator.begin(): await repo.lock_group_bindings(creator, data["group_id"]) # как `ConsolidationService.create` group = await creator.get(CompanyGroup, data["group_id"]) assert group is not None entities = [await creator.get(LegalEntity, entity_id) for entity_id in data["entity_ids"]] version = await _version(creator, group, [entity for entity in entities if entity is not None]) version_id = version.id confirm = asyncio.create_task(client.post(f"/documents/{data['document_id']}/confirm")) await _wait_for_lock_waiter(maker) assert not confirm.done() response = await asyncio.wait_for(confirm, LOCK_WAIT_SECONDS) assert response.status_code == HTTP_OK, response.text assert await _committed_bound(maker, version_id) == {data["document_id"]} async with maker() as session: assert await _status(session, version_id) is ConsolidationStatus.DATA_CORRECTION async def _committed_replacement(maker: async_sessionmaker[AsyncSession], data: dict[str, Any]) -> uuid.UUID: """У ЮЛ документа `data` — прежний файл с `CONFIRMED` квартальной ОСВ 60 за Q2 (тот же ключ автозамены, что у документа `data`, файл которого станет новым). Возвращает id прежнего документа.""" async with maker() as session, session.begin(): entity = await session.get(LegalEntity, data["owner_id"]) assert entity is not None return (await _confirmed(session, entity, quarter=Q2)).id async def _supersede(maker: async_sessionmaker[AsyncSession], data: dict[str, Any]) -> list[uuid.UUID]: """Автозамена в отдельной транзакции: новый файл — файл документа `data` (как в конце `map_file`).""" async with maker() as session, session.begin(): new_file = await session.get(StoredFile, data["file_id"]) new_document = await session.get(Document, data["document_id"]) assert new_file is not None assert new_document is not None replaced = await supersede_previous( session, new_file=new_file, new_documents=[new_document], uploader_id=None, now=utc_now() ) return [file.id for file in replaced] async def _replace_event(maker: async_sessionmaker[AsyncSession]) -> AuditEvent: async with maker() as session: stmt = select(AuditEvent).where(AuditEvent.action == AuditAction.FILE_REPLACE) return (await session.execute(stmt)).scalar_one() @pytest.mark.integration async def test_supersede_waits_for_pick_up_and_unbinds_replaced_document( committed_sessionmaker: async_sessionmaker[AsyncSession], ) -> None: """R-2, автозамена ∥ подхват: правка периметра держит версию, добавила ЮЛ и подхватила его `CONFIRMED` документ (ещё не зафиксировано). Автозамена файла этого документа блокирует надмножество версий ГК, ждёт правку и после её commit снимает уже видимую привязку; `unbound_version_ids` — эта версия. Блокируй автозамена только версии, где документ уже привязан, — она не увидела бы незафиксированную привязку и оставила бы её висеть.""" maker = committed_sessionmaker data = await _committed_setup(maker) old_document_id = await _committed_replacement(maker, data) async with maker() as editor, editor.begin(): version = await repo.lock_version(editor, data["version_id"]) assert version is not None consolidation = await editor.get(Consolidation, version.consolidation_id) assert consolidation is not None editor.add(ConsolidationMember(consolidation_version_id=version.id, legal_entity_id=data["owner_id"])) await editor.flush() assert ( await consolidation_documents.bind_confirmed( editor, version=version, consolidation=consolidation, entity_ids=[data["owner_id"]] ) == 1 ) replacement = asyncio.create_task(_supersede(maker, data)) await _wait_for_lock_waiter(maker) assert not replacement.done() replaced = await asyncio.wait_for(replacement, LOCK_WAIT_SECONDS) assert len(replaced) == 1 assert old_document_id not in await _committed_bound(maker, data["version_id"]) event = await _replace_event(maker) assert event.after_value is not None assert event.after_value["unbound_version_ids"] == [str(data["version_id"])] @pytest.mark.integration async def test_supersede_waits_for_create_and_unbinds_replaced_document( committed_sessionmaker: async_sessionmaker[AsyncSession], ) -> None: """R-2, автозамена ∥ создание версии: создание держит advisory ГК и подхватило `CONFIRMED` документ в новую (невидимую до commit) версию; автозамена ждёт advisory ГК и после commit снимает привязку новой версии.""" maker = committed_sessionmaker data = await _committed_setup(maker, spare=0, with_version=False) old_document_id = await _committed_replacement(maker, data) async with maker() as creator, creator.begin(): await repo.lock_group_bindings(creator, data["group_id"]) # как `ConsolidationService.create` group = await creator.get(CompanyGroup, data["group_id"]) assert group is not None entities = [await creator.get(LegalEntity, entity_id) for entity_id in data["entity_ids"]] version = await _version(creator, group, [entity for entity in entities if entity is not None]) consolidation = await creator.get(Consolidation, version.consolidation_id) assert consolidation is not None bound = await consolidation_documents.bind_confirmed( creator, version=version, consolidation=consolidation, entity_ids=data["entity_ids"] ) assert bound == 1 version_id = version.id replacement = asyncio.create_task(_supersede(maker, data)) await _wait_for_lock_waiter(maker) assert not replacement.done() await asyncio.wait_for(replacement, LOCK_WAIT_SECONDS) assert old_document_id not in await _committed_bound(maker, version_id) event = await _replace_event(maker) assert event.after_value is not None assert event.after_value["unbound_version_ids"] == [str(version_id)] # --- on_documents_mapped: NEEDS_CHECK при маппинге (osv-result-mapping, D-6) ------------------------------- async def _mapped_file_documents( session: AsyncSession, entity: LegalEntity, *, uploaded_by: uuid.UUID | None, status: DocumentStatus, quarter=Q2 ) -> tuple[StoredFile, list[Document]]: """Файл `PROCESSED` ЮЛ с одним квартальным документом ОСВ 60 за 2024 `quarter` в статусе `status`.""" file, job = await make_stored_file( session, entity=entity, status=FileStatus.PROCESSED, job_status=RecognitionStatus.done, uploaded_by=uploaded_by ) document = await make_document( session, file=file, job=job, account=ACCOUNT, year=YEAR, quarter=quarter, status=status, is_cumulative=False ) return file, [document] @pytest.mark.integration async def test_on_documents_mapped_moves_documents_processing_to_data_correction(seeded: AsyncSession) -> None: group, entities = await _group(seeded) version = await _version(seeded, group, entities, status=ConsolidationStatus.DOCUMENTS_PROCESSING) user = await _user(seeded) file, documents = await _mapped_file_documents( seeded, entities[0], uploaded_by=user.id, status=DocumentStatus.NEEDS_CHECK ) moved = await consolidation_documents.on_documents_mapped(seeded, file=file, documents=documents, now=utc_now()) assert moved == [version.id] assert await _status(seeded, version.id) is ConsolidationStatus.DATA_CORRECTION last = (await _history(seeded, version.id))[-1] assert (last.from_status, last.to_status, last.changed_by) == ( ConsolidationStatus.DOCUMENTS_PROCESSING, ConsolidationStatus.DATA_CORRECTION, user.id, ) [event] = ( await seeded.execute(select(AuditEvent).where(AuditEvent.action == AuditAction.CONSOLIDATION_STATUS_CHANGE)) ).scalars() assert event.user_id == user.id assert await _bound(seeded, version.id) == set() # привязку маппинг не создаёт @pytest.mark.integration @pytest.mark.parametrize("status", [ConsolidationStatus.INITIATED, ConsolidationStatus.DATA_CORRECTION]) async def test_on_documents_mapped_ignores_other_statuses(seeded: AsyncSession, status: ConsolidationStatus) -> None: group, entities = await _group(seeded) version = await _version(seeded, group, entities, status=status) user = await _user(seeded) file, documents = await _mapped_file_documents( seeded, entities[0], uploaded_by=user.id, status=DocumentStatus.NEEDS_CHECK ) history = len(await _history(seeded, version.id)) moved = await consolidation_documents.on_documents_mapped(seeded, file=file, documents=documents, now=utc_now()) assert moved == [] assert await _status(seeded, version.id) is status assert len(await _history(seeded, version.id)) == history @pytest.mark.integration async def test_on_documents_mapped_ignores_recognized_document(seeded: AsyncSession) -> None: group, entities = await _group(seeded) version = await _version(seeded, group, entities, status=ConsolidationStatus.DOCUMENTS_PROCESSING) file, documents = await _mapped_file_documents( seeded, entities[0], uploaded_by=(await _user(seeded)).id, status=DocumentStatus.RECOGNIZED ) moved = await consolidation_documents.on_documents_mapped(seeded, file=file, documents=documents, now=utc_now()) assert moved == [] assert await _status(seeded, version.id) is ConsolidationStatus.DOCUMENTS_PROCESSING @pytest.mark.integration async def test_on_documents_mapped_skips_document_that_does_not_fit_version(seeded: AsyncSession) -> None: """Квартальный документ Q4 не подходит версии Q3 (D-5): `NEEDS_CHECK`, но перехода нет.""" group, entities = await _group(seeded) version = await _version(seeded, group, entities, status=ConsolidationStatus.DOCUMENTS_PROCESSING) file, documents = await _mapped_file_documents( seeded, entities[0], uploaded_by=(await _user(seeded)).id, status=DocumentStatus.NEEDS_CHECK, quarter=Q4 ) moved = await consolidation_documents.on_documents_mapped(seeded, file=file, documents=documents, now=utc_now()) assert moved == [] assert await _status(seeded, version.id) is ConsolidationStatus.DOCUMENTS_PROCESSING @pytest.mark.integration async def test_on_documents_mapped_skips_entity_outside_perimeter(seeded: AsyncSession) -> None: group, entities = await _group(seeded, members=3) version = await _version(seeded, group, entities[:2], status=ConsolidationStatus.DOCUMENTS_PROCESSING) file, documents = await _mapped_file_documents( seeded, entities[2], uploaded_by=(await _user(seeded)).id, status=DocumentStatus.NEEDS_CHECK ) moved = await consolidation_documents.on_documents_mapped(seeded, file=file, documents=documents, now=utc_now()) assert moved == [] assert await _status(seeded, version.id) is ConsolidationStatus.DOCUMENTS_PROCESSING @pytest.mark.integration async def test_on_documents_mapped_without_uploader_logs_and_skips(seeded: AsyncSession) -> None: """`uploaded_by` NULL: `changed_by NOT NULL` — переход пропускается, в лог пишется причина, ошибки нет.""" group, entities = await _group(seeded) version = await _version(seeded, group, entities, status=ConsolidationStatus.DOCUMENTS_PROCESSING) file, documents = await _mapped_file_documents( seeded, entities[0], uploaded_by=None, status=DocumentStatus.NEEDS_CHECK ) messages: list[str] = [] sink = logger.add(lambda message: messages.append(str(message)), level="INFO") try: moved = await consolidation_documents.on_documents_mapped(seeded, file=file, documents=documents, now=utc_now()) finally: logger.remove(sink) assert moved == [] assert await _status(seeded, version.id) is ConsolidationStatus.DOCUMENTS_PROCESSING assert any(str(file.id) in message and "uploaded_by" in message for message in messages) async def _mapped_in_own_transaction(maker: async_sessionmaker[AsyncSession], data: dict[str, Any]) -> list[uuid.UUID]: """Хук маппинга файла `data["file_id"]` в своей транзакции (как цикл маппинга: файл и документы уже есть).""" async with maker() as session, session.begin(): file = await session.get(StoredFile, data["file_id"]) document = await session.get(Document, data["document_id"]) assert file is not None assert document is not None return await consolidation_documents.on_documents_mapped( session, file=file, documents=[document], now=utc_now() ) @pytest.mark.integration async def test_on_documents_mapped_waits_for_member_edit_and_sees_new_member( committed_sessionmaker: async_sessionmaker[AsyncSession], ) -> None: """R-2: правка периметра держит версию `DOCUMENTS_PROCESSING` и добавила ЮЛ файла (не зафиксировано); хук маппинга блокирует надмножество версий ГК, ждёт правку и после commit видит нового члена — переход не теряется.""" maker = committed_sessionmaker data = await _committed_setup(maker) async with maker() as session, session.begin(): version = await session.get(ConsolidationVersion, data["version_id"]) assert version is not None version.status = ConsolidationStatus.DOCUMENTS_PROCESSING file = await session.get(StoredFile, data["file_id"]) assert file is not None file.uploaded_by = data["user_id"] async with maker() as editor, editor.begin(): assert await repo.lock_version(editor, data["version_id"]) is not None editor.add(ConsolidationMember(consolidation_version_id=data["version_id"], legal_entity_id=data["owner_id"])) await editor.flush() hook = asyncio.create_task(_mapped_in_own_transaction(maker, data)) await _wait_for_lock_waiter(maker) assert not hook.done() moved = await asyncio.wait_for(hook, LOCK_WAIT_SECONDS) assert moved == [data["version_id"]] async with maker() as session: assert await _status(session, data["version_id"]) is ConsolidationStatus.DATA_CORRECTION @pytest.mark.integration async def test_on_documents_mapped_waits_for_transition_to_documents_processing( committed_sessionmaker: async_sessionmaker[AsyncSession], ) -> None: """R-2/№40, маппинг ∥ загрузка (`after_upload_to_version`): версия `INITIATED` держится транзакцией, которая переводит её в `DOCUMENTS_PROCESSING` (не зафиксировано). Хук блокирует надмножество версий «в работе» (а не только `DOCUMENTS_PROCESSING`), поэтому встаёт в ожидание строки версии и после commit видит `DOCUMENTS_PROCESSING` — переход в `DATA_CORRECTION` не теряется. Хук, ищущий версии только в `DOCUMENTS_PROCESSING`, снимок `INITIATED` пропустил бы без ожидания и вернул бы `[]`.""" maker = committed_sessionmaker data = await _committed_setup(maker, members=2, spare=0) # владелец документа — член периметра, версия INITIATED async with maker() as session, session.begin(): file = await session.get(StoredFile, data["file_id"]) assert file is not None file.uploaded_by = data["user_id"] async with maker() as uploader, uploader.begin(): version = await repo.lock_version(uploader, data["version_id"]) assert version is not None assert version.status is ConsolidationStatus.INITIATED version.status = ConsolidationStatus.DOCUMENTS_PROCESSING await uploader.flush() hook = asyncio.create_task(_mapped_in_own_transaction(maker, data)) await _wait_for_lock_waiter(maker) assert not hook.done() moved = await asyncio.wait_for(hook, LOCK_WAIT_SECONDS) assert moved == [data["version_id"]] async with maker() as session: assert await _status(session, data["version_id"]) is ConsolidationStatus.DATA_CORRECTION