The project aims to create a Machine Learning workflow that helps TV planners in their work using data from multiple sources.

• Deploy Machine Learning models to production workflow.
• Build reliable and scalable Machine Learning workflows.
• Build reliable data ingestion and processing pipelines.
• Reduce model response time for real-time predictions.
• Build a modern Python package release process.
• Optimize CI/CD pipelines and container image building.
• Integrated code quality checks.
• Build CI pipeline for automated Python package releases to a private registry.

Fix, optimize, and improve daily data processing pipelines that suffered from reliability issues and performance bottlenecks. The changes improved data decision-making and developer productivity

• Identify performance issues and prepare an optimization roadmap.
• Fix data pipeline bottlenecks.
• Integrate automated testing and CI/CD pipelines.
• Integrated code quality checks.

Initially, the data infrastructure was unreliable, with frequent interruptions and delays in data delivery. In this project, we reworked the data infrastructure, data processing, and data storage to improve reliability and performance.

• Upgrade Airflow deployment and introduce automated deployments on Kubernetes with Helm.
• Implement logging to keep track of events in data pipelines.
• Refactor data processing and introduce partitioning.
• Decouple processing power and data storage to make data available earlier.
• Introduce unit, integration and end-to-end test automation and code formatting.

After developing a mobile application, the company extracts value from the collected data using technologies like Spark, Airflow, MLflow. Intelligent end-to-end solutions are provided to stakeholder requirements.

• Validate and integrate in the architecture, data ingestion frameworks like Airbyte.
• Implement automation and ML model training orchestration.
• Serve ML models using serverless infrastructure.
• Automate data ingestion, and model deployment.
• Unit, integration, and end-to-end test automation.

Using Machine Learning and NLP techniques, to infer what the customers are not buying for the supplier and target these gaps with personalized sales promotions. The product is used to assist Sales Manages in increasing basket sizes for their customers.

• Starting from locally running MVP, move to a production solution running on Kubernetes on GCP Cloud.
• Lead implementation of incremental performance improvements, to fit the demands of production load.
• Spearheaded the implementation of modern ML application deployment technologies and frameworks to the project.
• Automate data ingestion, model training, evaluation and deployment.
• Unit, integration and end-to-end test automation.

In stock exchange companies, the volume of collected data increases daily. Also market value for these data increases. Considering these dynamic features of the project scope, we created an architecture where historical data could be provided to users and allow them to enrich this data in their own scope.

• Creating an architecture for fast and cost-efficient access to historical data.
• Development of data transformation strategy to persist extensions of precalculation layer.
• Development of schema evolution strategy.

Project aims to create a decoupled architecture to process and persist data on a daily basis, collected into GCP. Also, in the scope of this project, we defined the CI/CD strategy for all the components of the data pipeline.

• Create an ingestion layer to make data available in BigQuery on a daily basis.
• Create the CI/CD pipeline for all components.
• Spearheaded the implementation of modern ML application deployment technologies and frameworks to the project.
• E2E-Integration testing (design, data generation, execution)
• Documentation

Create a self-service framework for ingesting data from different sources and preprocessing, keeping in mind requirements from the end consumer of these data. The framework is designed to work in a cloud-native environment. A set of HTTP Rest APIs was made available to users to provide necessary functionality around on-demand AWS EMR clusters.

• Development of data transformation and aggregation algorithms (Scala) to facilitate new load types.
• Development of the metadata management and data transformation orchestration application, written in Python, to support new destination environments.
• Development of unit, integration, and end-to-end test for all components of the application, Scala and Python.

• Implemented a Big Data Platform for ingestion, storage, and processing of batch and real-time data from a variety of data sources within the customer. 
• As part of that, a centralized search engine was designed and implemented for company-wide knowledge and documentation of data sources. 
• In another project, we aimed to transfer the persistence layer of one analytic application from Oracle to Impala. 
• Data was provided on a regular basis, and the result of processing with Spark and Scala was a set of tables with more than 10000 columns. 
• These jobs were scheduled on Apache Airflow.

• Full-Stack software Developer in several projects used by up to 1500 users.