API Response Time Down 80% for a Bloomberg Rival
80%
Less API response time67%
Faster feature delivery time2х
Faster background tasks
Client Overview
Business Problem
Situation Before
Core Pain
Risks
- Latency and reliability failures on a real-time financial platform would undermine user trust and platform viability
- Without proper monitoring, any problems with the live platform could remain undetected, compromising uptime and data integrity
- Slow delivery velocity would risk the platform falling behind market and regulatory timelines
Constraints
- Short timelines for releasing new features are guided by the product roadmap and market needs
- Work involves multiple teams, including backend, frontend, system analysts, architects, and DevOps, requiring close coordination
- There is a need to process data quickly in real-time for exchange feeds, news and analytics, with strict latency expectations
- The work must also meet EU rules for handling financial data and other relevant securities market laws
Project Goals
- Create a practical European-based alternative to Bloomberg for both institutional and retail financial analytics users
- Speed up the development of new features to keep up with the product roadmap and market demand
- Set up a dependable, high-availability platform that can serve real-time financial data at scale
- Develop and implement a microservices architecture for independent service scalability
- Establish real-time pipelines using WebSocket and asynchronous processes for news, trading data and market data
- Optimize API response time and background task throughput for financial-scale workloads
- Set up monitoring, alerts and load testing systems to ensure stability of the system
- Integrate with external data sources including regional exchange feeds and news aggregators
- API response time materially reduced from baseline
- Background task processing speed doubled
- Overall instance uptime improved by over 20%
- Feature delivery cycle reduced from 12 weeks to under 4 weeks
Solution
The platform runs on a service-oriented architecture for news, market data, exchange trades, financial models, and dashboards, where KrakenD serves as the API gateway and handles inter-service routing. Kafka is used for high-throughput event streaming, while RabbitMQ is utilized for task-based messaging, with aiokafka and aio-pika as asynchronous consumers. ClickHouse hosts exchange trade storage and analytics, chosen for its performance on high-volume time-series data. MongoDB stores dashboard configurations as widget JSON, and PostgreSQL is the primary structured data store.
Pynest developed domain-specific microservices for News Processing, Market Data Storage and Retrieval, Exchange Trade Parsing and Financial Model Calculation. The team optimized SQL querying and implemented Redis caching for a set of queries, which brought API response time down 80%. Pynest also introduced system-wide monitoring and alerts across the platform.
The technologies used for containerization and orchestration are Docker and Kubernetes. The reverse proxy is provided by Nginx. The automated pipelines run on GitLab CI/CD and Jenkins. For file and data persistence, cloud object storage is used. Operations automation is done through bash scripting. Monitoring and alerting are provided by Prometheus and Grafana.
The Keycloak service manages all aspects of identity and access management on the platform, including authentication and authorization. Token management and encryption are done through PyJWT and cryptographic libraries. The platform complies with EU financial regulations and securities legislation, including the data residency requirements for European client data.
Key Steps
Pynest engineers joined the client cross-functional delivery teams under a team augmentation model. Following knowledge transfer and architecture review, the team aligned on development processes, set up collaboration across the backend, frontend, system analyst and DevOps teams, and began working within the client Scrum ceremonies, GitLab CI/CD pipelines and architectural decision process.
The team designed and implemented domain-specific microservices for news processing, market data storage, exchange trade parsing and financial model calculations. The event-driven architecture was built on Kafka and RabbitMQ, while ClickHouse, PostgreSQL and MongoDB were integrated to support high-volume financial data processing and configurable analytical dashboards.
As the platform matured, engineering shifted toward performance and reliability. SQL queries were optimized with Redis caching, monitoring and alerting were introduced with Prometheus and Grafana, and load testing became part of the delivery process. These changes cut API response time by 80%, doubled background task throughput and improved overall platform stability.
With the platform running on a stable microservices foundation, the client expanded development into more analytics capabilities, dashboard functionality and exchange integrations. Cross-team practices, including regular architecture reviews and product demos, shortened the feature delivery cycle from 12 weeks to 4 while keeping the product roadmap on track.
Pynest keeps enhancing the platform, expanding support for more exchanges, financial instruments and analytical models. The modular architecture lets new capabilities land without reworking the existing services.
Results & Impact
Project Snapshot
See Other Works
Secure LLM Gateway for a 63,000-Person Enterprise
Secure enterprise AI access with centralized governance, lower latency, content controls, and full token usage visibility
Bond Settlement Cut from 2 Days to 30 Minutes
End-to-end capital markets platform automating bond issuance, trading, settlement, and investor servicing
Power Station Analytics: Algorithm Suite Delivered
Real-time equipment analytics platform for a power station with automated calculations, monitoring, and reliable fault detection
