Financial-data processing platform
Smart Transformer
A full-stack platform that modernized spreadsheet-driven financial-data workflows through structured ingestion, normalization, deterministic processing, and bounded AI-assisted analysis.
Engineering areas
- Backend engineering
- Full-stack integration
- Software architecture
- Data processing
- AI-assisted systems
Context
Project setting
Smart Transformer was developed during a six-month final-year engineering internship in 2026 for a financial-services organization.
The case study stays at the software-system level: it describes how a spreadsheet-driven workflow became a structured application without exposing the organization or its internal operations.
Problem
System problem
Spreadsheet/VBA and semi-manual workflows needed a maintainable path from heterogeneous inputs to normalized data, repeatable calculations, analysis, and structured outputs.
- 01
Ingest and validate heterogeneous spreadsheet inputs.
- 02
Normalize data before calculation and analysis.
- 03
Keep deterministic operations reproducible.
- 04
Generate structured JSON and XML outputs.
Constraints
Design boundaries
The platform had to account for heterogeneous input files, sensitive data, predictable calculations, authentication, traceable processing, and file-based interfaces.
- C01Heterogeneous spreadsheet inputs
- C02Deterministic calculation requirements
- C03Sensitive data handling
- C04Authentication requirements
- C05Traceable structured processing
- C06File-oriented interfaces
My Scope
Contribution
Contributed substantially to the design and implementation of the platform across backend services, frontend integration, data normalization, deterministic processing, authentication, AI orchestration, automated testing, and delivery automation.
- Backend services and APIs
- Frontend integration
- Data normalization and deterministic processing
- Authentication and AI orchestration
- Automated testing and delivery automation
System Architecture
Ports and adapters
A React interface communicates with a FastAPI application/API. Hexagonal boundaries separate the application and domain-processing core from persistence, identity, file, and AI adapters.
Domain and application logic remain separate from infrastructure adapters.
External interfaces can evolve without redefining the processing core.
Clear boundaries reduce coupling and make core behavior easier to test.
Figure A
Ports and adapters view
- 01React web interface
- 02FastAPI application / API
- 03Application and domain-processing core
Ports / adapters boundary
- PostgreSQL persistence adapter
- Versioned reference-data adapter
- Identity adapter
- File input/output adapters
- AI orchestration adapter
Data Workflow
Processing sequence
Excel or structured input moves through ingestion, validation, normalization, a structured JSON representation, deterministic processing, bounded analysis, and XML or structured output generation.
Figure B
Processing path
- 01Excel / structured input
- 02Ingestion and validation
- 03Normalization
- 04Structured JSON representation
- 05Deterministic processing
- 06AI-assisted analysis where applicableBounded, probabilistic stage
- 07XML / structured output
Engineering Decisions
Decision record
The central decisions concerned deterministic and probabilistic boundaries, domain isolation, normalized internal data, versioned reference data, browser-oriented authentication, and repeatable validation.
D01 Deterministic processing versus AI assistance
- Decision
- Keep structured business-rule calculations in deterministic Python logic and use AI for analysis and interpretation-oriented tasks.
- Reason
- Calculations requiring predictable, reproducible behavior should not depend on probabilistic output.
- Outcome
- AI augments the workflow without becoming authoritative for deterministic calculations.
D02 Hexagonal architecture
- Decision
- Separate application and domain-processing logic from persistence, identity, file, and AI integrations through ports and adapters.
- Reason
- The processing core needed clear boundaries from infrastructure concerns.
- Outcome
- Adapters remained replaceable and core behavior could be tested with less infrastructure coupling.
D03 Normalized JSON representation
- Decision
- Normalize heterogeneous spreadsheet data into a structured JSON representation before downstream processing.
- Reason
- A stable internal representation gives calculation, analysis, and output stages a consistent input boundary.
D04 Versioned reference data
- Decision
- Manage mapping and reference configuration as version-aware data without exposing internal values.
- Reason
- Processing needed controlled reference changes while retaining historical context.
D05 Keycloak with PKCE and JWT
- Decision
- Use browser-oriented Authorization Code Flow with PKCE S256 and validate JWTs with RS256.
- Reason
- Authentication needed a standards-based browser flow rather than direct credential handling in the application.
D06 Automated tests and delivery validation
- Decision
- Use pytest, Tox, GitHub Actions, and Docker for repeatable tests, validation, and delivery preparation.
- Reason
- A multi-stage processing system benefits from regression protection and reproducible automation.
AI Integration
Bounded capability
AI-assisted agents supported field identification, analysis-oriented tasks, and report generation. They augmented the workflow without becoming authoritative for deterministic business calculations.
Deterministic
Authoritative processing
Structured business-rule calculations remain predictable and reproducible.Probabilistic
Analysis assistance
AI supports interpretation-oriented work within an explicit boundary.Agent responsibilities
- A01Field identification
- A02Data analysis
- A03Calculation-analysis assistance
- A04Report generation
Security
Browser authentication
Keycloak provided OAuth 2.0 and OpenID Connect authentication through Authorization Code Flow with PKCE S256, with JWTs validated using RS256.
This keeps direct credential handling out of the application flow while the API verifies signed access tokens. Configuration, realm details, client identifiers, and private endpoints are intentionally excluded.
Delivery & Quality
Validation and delivery
More than 450 automated tests, pytest, Tox, GitHub Actions, and Docker supported regression checks, automated validation, and repeatable delivery preparation.
- pytest for automated test execution
- Tox for repeatable validation environments
- GitHub Actions for CI validation
- Docker for containerized delivery preparation
Results
Measured outcome
For the tested workload, a workflow that previously took roughly four hours was reduced to under 45 seconds.
For the tested workload
Engineering Takeaways
Engineering record
The project reinforced the value of normalizing heterogeneous inputs early, isolating domain logic, designing replaceable adapters, and keeping probabilistic assistance outside deterministic authority boundaries.
- T01
Normalize heterogeneous inputs before domain processing.
- T02
Make deterministic and probabilistic boundaries explicit.
- T03
Isolate infrastructure so external integrations remain replaceable.
- T04
Use automated tests and repeatable delivery to reduce regression risk.
Technology Index
Technical reference
A compact reference to the verified technologies used across interface, backend, data, security, architecture, AI, quality, and delivery.
- Frontend
- React / TypeScript
- Backend
- Python / FastAPI / Pydantic / SQLAlchemy / Alembic
- Data
- PostgreSQL / Structured JSON / Excel processing / XML
- Security
- Keycloak / OAuth 2.0 / OIDC / PKCE S256 / JWT / RS256
- Architecture
- Hexagonal architecture / Ports and adapters
- AI
- LLM-based agents / Multi-agent orchestration
- Quality
- pytest / Tox / Automated testing
- Delivery
- GitHub Actions / Docker / CI/CD