The uploader took a single .zip, so a day split across several archives, or a
handful of loose XML files, needed one run each.
- main.py and scripts/local_zip_test_app.py now use accept_multiple_files and
take .xml alongside .zip and the tar family. Each attachment is written to
its own temp subdirectory, because two attachments may share a name.
- run_batch takes input_paths instead of zip_path.
- New app/pipeline/attachments.py flattens the attachments into a list of XML
files. Archive format is detected from the content rather than the suffix,
since exports arrive with the wrong extension often enough. .rar and .7z are
recognised only to explain why they are refused.
- A broken attachment no longer aborts the run. It becomes one protocol row
and the remaining attachments are still processed; the reason is also shown
in the UI, not only in the protocol file.
- Files repeated across attachments are processed once and the repeats are
recorded with status "skip". Attaching the same archive twice would
otherwise double every row of the report.
- Archives are unpacked member by member with the target path checked against
the extraction directory. tarfile.data_filter only exists from Python 3.11.4
and extractall would otherwise let a "../" member escape.
- The protocol gains a "Вложение" column, so it is clear which attachment a
file came from when names repeat across archives.
Checked end to end on the 03.02.2023 export: six attachments (two archives,
two loose XML files, one broken archive, one duplicate archive) give 57 files
processed, 9 repeats skipped, one attachment refused with its reason, and the
report is still produced.
The customer's ruling of 24.09.2026 is that every value present in the XML
must reach the report, taking into account which block holds it. Two defects
broke that for operation code 5016 with transfer types 4 and 5.
Wrong paths in mapping_table.json. Columns 48-63 spelled the block as
"СведенияПереводыДс" and columns 49-59 spelled the address as
"АдрМестоПриемаВыдача"; the schema uses "СведенияПереводыДС" and
"АдрМестаПриемаВыдача". Those columns carry allow_short_lookup=false, so they
can only match on a fully scoped path and never matched at all: the whole cash
receipt block (AV-BI) and the transfer status (BJ) stayed empty on every file.
The existing test passed only because it fed the same misspellings.
Rules that deleted data instead of filling it:
- Rules 38, 44-45 and 46 wiped the account columns for code 5016. The XML of
those records carries СчетБанкПолучатель = 00000000000000000000, which is
exactly the value the customer asked for. The rules now only substitute the
placeholder into an empty column, and never for 5016.
- Rule 44 also wiped the correspondent accounts for operator types 3 and 5
even when the XML had them.
- The cashless rule wiped columns 50, 61, 65, 76, 84, 95, 99 and 101 for
transfer types 1-4, 7 and 10-14, although type 4 is by definition a transfer
that accepts cash and must carry the receipt block.
- Rule 62 limited the transfer status to types 2, 5 and 8.
- Rule 30 limited the metal name to code C99, while the available exports only
ever carry code A90.
Verified on the sample exports: all 16 records with code 5016 now carry the
zeros in column 45, the real account in column 46, and no invented
placeholders in columns 38 and 44.
- Draw a grid on the two header rows only, data cells stay borderless.
- Align column names to the top so the start of a long name stays visible.
- Narrow the 14 code columns to the length of their values; the rule matches
by column name, so identical indicators in every block get equal widths.
- Halve both payment system name columns (32 -> 16).
- Give "Характеристика операции" the same width as "Назначение платежа".
- Drop the group title above "Статус перевода": it duplicated the column name.
The group cell stays in place so the header grid is not broken.
- Pass the cell format to write_string. Identifier columns (БИК, ИНН, ОГРН,
КПП, СВИФТ, account numbers) were written without one, so text wrapping and
vertical alignment never applied to them - this is why БИК had no line break.
The width padding and the header rotation are single constants: the customer
has not confirmed yet whether "N знаков" means the value length (implemented)
or an amount to subtract, and whether narrow headers should be rotated.
The АдрМестаПриемаВыдача block under СведенияКартаИнБанк (code 1009) has no
single АдресСтрока tag, unlike the same-named block under
СведенияВнесениеПолучениеНалДС — column 99 was mapped directly onto that
nonexistent tag and stayed empty. It now assembles the value from the
individual address components (Индекс, КодОКСМ, КодСубъектаПоОКАТО, Район,
Пункт, Улица, Дом, Корп, Оф), the same way column 153 already does for
registration place. Also disabled allow_short_lookup for this column, since
it was previously able to pick up a same-named tag (e.g. Улица) from an
unrelated address elsewhere in the file.
Added regression tests at both the mapping level and a full parse_xml_content
-> build_fixed_row_by_index pipeline level with a synthetic code-1009 XML.
Each call into opencloud_token_store is wrapped in its own asyncio.run()
(app/app_config.py, main.py's token popover), so every call gets a fresh
event loop. The asyncpg connection pool is bound to the loop it was created
in, so the next asyncio.run() could try to reuse a connection from an
already-closed loop and fail — unrelated to actual DB availability, but the
broad except Exception in resolve_opencloud_access_token() silently
swallowed it and fell through to "token not found", masking the real cause.
- add_token/get_token/remove_token/list_usernames now call engine.dispose()
in a finally block so each asyncio.run() starts with a clean pool.
- resolve_opencloud_access_token() now surfaces the real exception via
st.error() before falling back to the .env token (kept only for local dev
without a DB).
- Added tests/unit/test_opencloud_token_store.py covering that dispose() is
called after every store operation, including when commit() raises.
- Drop SMB/SAMBA subsystem completely: app/samba_access.py, app/path_settings.py,
app/path_store.py, app/pipeline/file_manager.py and their tests, plus the
smbprotocol dependency and all related UI (login/password fields, path editor,
SMB diagnostics) in main.py.
- run_batch now takes a .zip attached via Streamlit's file_uploader (same as the
local dev tool) instead of reading from SMB or a fictitious OpenCloud "Attach"
folder; renamed from run_local_zip_batch since it's now the single canonical
batch entrypoint used by both main.py and scripts/local_zip_test_app.py.
- Fixed run number resolution: added next_opencloud_run_number() which lists the
Out/year/month/day folder via WebDAV PROPFIND, so repeated runs land in
incrementing subfolders instead of always overwriting "1".
- Added a self-service "Добавить токен" popover in main.py so users can link
their own OpenCloud App Token to their Keycloak account (opencloud_token_store)
without an admin running the CLI script; reworded the missing-token error to
point at it.
Files no longer go into a shared test space with a guessed folder name.
Instead, resolve each user's personal drive id via GET /graph/v1.0/me/drives
(Basic Auth, same App Token as uploads) on every run, and upload into
Out/год/месяц/день/номер_запуска inside that drive, creating the path via
MKCOL since a fresh personal drive won't have it yet.
Removes the now-obsolete OPENCLOUD_SPACE_ID / resolve_opencloud_user_folder
and the shared-space smoke script.
