Principal Data Engineer Postgres, Spark, python, scala, AWS, schema and ETLs fine-tuning
Aktualisiert am 09.10.2024
Profil
Freiberufler / Selbstständiger
Remote-Arbeit
Verfügbar ab: 15.10.2024
Verfügbar zu: 100%
davon vor Ort: 100%
Python
Apache Spark
PostgreSQL
Scala
C++
Elastic Search
Apache Cassandra
Redis
Kafka
AWS
Terraform
Kubernetes
Helm
CI/CD
Data-Mart
Datawarehouse
Applikationsarchitektur

Einsatzorte

Einsatzorte

Deutschland, Schweiz, Österreich
möglich

Projekte

Projekte

2024-10 - now

prepare system to scale, Performance tuning

principal engineer MongoDB Capacity Management Performance-Management ...
principal engineer

Architecture review of code C# and schema for MongoDB.

Prepare recommendation on schema design to scale from 7 billions rows to 14 billions: partition data, choosing hash functions to avoid collisions, introduce cache and queue to scale processing, address security implications for PII data salting, encryption, data anonimisation.


MongoDB Capacity Management Performance-Management Performance Measurement Stresstest
2 months
2024-08 - 2024-09

Audit of software development processes, decrease time-to-market

Principal engineer Changemanagement Continuous Delivery Continuous Integration ...
Principal engineer

Review and analyse existing development practices and processes

identify inefficiencies and bottlenecks

prepare a roadmap to improve time-to-market: optimise CI/CD processes, introduce autotests, perform vertical task slicing to facilitate working in small batches, prepare RFC/ADR before hand to align team on software design

Changemanagement Continuous Delivery Continuous Integration Testautomatisierung DevOps Azure Devops C#
CMO Group
2 months
2024-06 - 2024-07

Performance tuning - 2k read/writes per second - postgres, Django, python

PostgreSQL Django Python ...

Analyse Postgres data access patterns: read vs writes, hot tables, redundant and missing indexes, table and index sizes.

Check analyze and vacuum stats.

Collect query statistics - number of calls, mean and max time.

Review app architecture - job's queue, webhook and user triggered endpoint + kronjobs in kubernetes.

Collect metrics in prometheus and grafana related to db usage and code execution to map CPU spikes with suspicious code.

Together with engineering team run several sessions of code optimisation's that allow to improve overall performance 3 times.

Document runbooks and roadmap to prepare codebase for further growth.  

PostgreSQL Django Python Redis
Rakt technologies
2 years 6 months
2022-01 - 2024-06

Build a Uber-like match-making platform for cleaners and home-owners

CTO PostgreSQL Python Elastic Search ...
CTO

Tech co-founder:
  • Scale cross functional team from 0 to 12, during last 3 years - zero employee's churn
  • Decrease average time to market from week to 2 days
  • Small batches, automate everything, product mentality, extreme ownership, mob-programming and KISS principles, Agile (Scrum/Kanban), Kaizen, TDD,
  • Define and help to execute tech strategy to meet business needs - budgeting, risks vs priorities, tech stack and architecture, build vs buy


Create CleanBee - Uber-like platform to connect homeowners with cleaners:

  • Build initial version of backend Rest API (Python, FastAPI, pydantic, alembic, sqlalchemy, asyncio, mypy, pytest, psycopg, firebase, pre-commit, ruff)
  • Build initial version of SPA - web frontend (NextJS, javascript, typescript, tailwind, css)
  • Develop match-making search/recommendation engine and wrap it behind Rest API - feature engineering, core engine - KNN + ALS (scikit, redis, numpy, pandas)
  • Design data pipelines to analyze and create dashboards for conversion funnels (Elasticsearch, Google Query, PostgreSQL, Firebase analytics, data governance, data cleansing, data quality)
  • Setup CI/CD and monitoring (Github actions, terraform, k8s, helm, vault, grafana, prometheus, ELK, GCD, AWS, docker, PostgreSQL, fastlane)
  • Integration of social signing, analytics, support chat and payment into mobile app (MVVM, flutter, clean architecture, getstream, firebase notification deeplinks)
  • UI & integration tests (curated env with auto-generated dummy data + testproject, apium, patrol, firebase, test project)


Create MVP app (iOS & Android) to control facial expressions - on device computer vision pipelines:

  • Port C++ core engine from x86_64 to arm64 (cross-compilation, cross-toolchain)
  • Integrate C++ into native code (jni, CMake, ninja, bazel)
  • Convert tensorflow model to tensorflow lite format (knowledge distillation, quantisation, prunning)
  • To speedup prototyping and algorithm assessment - create an demo pipeline to experiment with various model?s architecture and pre/post processing (python, javascript, react, flask, tensorflow, tensorboard, CVAT, DVC) 

AWS Google Cloud Azure
PostgreSQL Python Elastic Search AWS GCD Azure Hive AWS: EC2 EMR ECS S3 K8S Terraform c++ Kotlin Swift javascript tensorflow Docker github actions for CI\CD AWS Sagemaker Mechanical Turk DVC
Syzygy AI LLC
Berlin
2 years 8 months
2019-06 - 2022-01

Data infra cloud migration

senior data engineer Apache Spark Scala Python ...
senior data engineer
First senior engineer in data department - help company transition towards Data Mesh topology:
  • Migrate number of data pipelines from Cloudera private hosting environment into AWS cloud to support exponential demand for growth during Covid
  • Develop framework to assure data quality of digested events
  • Develop new data pipelines and extend existing to fulfill stackholder?s needs
  • Participate in on-call rotation to support SLA for business critical datasets
  • Performance tuning of data pipelines: code of ETLs, Spark & HDFS internals
AWS
Apache Spark Scala Python AWS Terraform Kubernetes Hive Impala Hadoop Spark stream processing Airflow Parquet AWS: EC2 EMR S3 DMS Sage Maker K8S
Hellofresh
Berlin
1 year 1 month
2018-06 - 2019-06

Manage development team of 3 developers

Co-founder & principal software engineer microservices application containerisation Python ...
Co-founder & principal software engineer
Manage development team of 3 developers:
  • Manage priorities and define project?s milestones
  • Gather and convert functional requirements to sprint?s tasks
  • Run code review and pair-coding sessions
  • Provide mentorship and assistance during criticals


Create automated crypto currency-trading system:

  • Develop scalable trading platform - high throughput low-latency fully automated trading engine for multi-agent deployments
  • Design reactive architecture to unify integration of additional exchanges
  • Lead implementation of unified UI for stakeholders:
  • rules management
  • deployment automation and monitoring
  • trend exploration and data discovery
  • Create dedicated services for live data stream acquisition and real time signaling
  • Inference engine for models backtesting
microservices application containerisation Python gevent SOA redis based event queues aws stream processing websockets postgres flask telegram bot api
on request
2 years 8 months
2015-11 - 2018-06

Build analytical project portfolio

senior software architect Spark Cassandra Zeppelin ...
senior software architect
Senior Architect:

  • Design, build and optimize high throughput highly available multi component data processing pipelines (up to 80k events per seconds, depending on cluster configuration).
  • Re-factor of business critical components of various products across the whole stack
  • Cassandra, Kafka, Elastic performance tuning - schema adjustments & implementation of high performance multithreaded shooters in C++, jvm & linux kernel optimization
  • Containerization of applications: maintaining internal docker registry, automate orchestration of dockers vms, docker performance tuning
  • Introduce company for modern SDLC practices and help to integrate it in a way to be coherent with business goals: continuous integration, devops, integration tests & auto-tests
  • Participate in business critical activities: sizing and hardware planning, develop POC for pre-sales events, requirements assessments, investigation of disaster incidents, etc
  • Develop and run internship program: software development, data analysis.


Head of analytics department:

  • Add several key functionalities for company's products portfolio.
  • Implement various analytical modules on top of Apache Spark.
  • Build a toolbox for algorithms assessments for data discovery and quick hypothesis checks
  • Assessment and adaptation of methods of AI\ML for such tasks as predictions\clusterization.
  • Application of methods Natural Language Processing (NLP) according to business goals.


Achievements:

  • Decrease storage requirements on 50%. Increase overall system performance from state when the full cluster cant deal with peak load to situation when ? of cluster?s nodes weren?t available due to power outage new system handle load without hiccups. 
  • Develop proof of concept application that lead to signing three new contracts with major clients for the whole company history.

Spark Cassandra Zeppelin postgres redis kafka akka scala Java SOA microservices reverse engineering application of math statistics data mining predictive analysis & DSP C++ boost python scikit docker
Security company
Abu Dhabi
11 months
2015-01 - 2015-11

Define and control execution of sprints for freelancers

Senior software engineer Algorithms machine learning postgres ...
Senior software engineer
Perform duty of CTO (july - september):
  • Manage hiring for backend/frontend/system administrator in order to meet project?s deadline
  • Define and control execution of sprints for freelancers
  • Provide technical leadership and coaching for team


Architect a data mining sub-system. Design, build and optimize search/recommendation engine:

  • Develop general architecture of server components, choose technology stack for
  • implementation, design database structure and implement middleware layer for interaction with 3rd party systems
  • Build web-crawler for machine learning algorithms evaluation
  • Perform evaluation of various machine learning techniques, embed most promising approach to general pipeline
  • Adapt core of recommendation engine for high-load project by profile and optimize
  • bottlenecks
  • Define and manage Mesos cluster deployment strategy
  • Control and manage of integration various clients (mobile/web) to backend services


Achievements:

  • Accuracy of search above 85 percent, time of reply less than 250 ms

Algorithms machine learning postgres elasticsearch redis scikit scrapy nltk gensim selenium Mesos cluster stack AWS stack vagrant docker git django SOAP RESTfull api
Insydo
Dubai
2 years 8 months
2012-05 - 2014-12

Develop application for 3d-scene reconstruction from RGB-D cameras

CTO/founder C++ OpenCL OpenCV ...
CTO/founder
Develop application for 3d-scene reconstruction from RGB-D cameras:
  • Porting reference kinect fusion implementation from CUDA, PCL library to OpenCL
  • Collaborate with PCL community to contribute to KinFu sub-module in order to overcome limitations of existing implementation: integrate RGB information to resulting point cloud, cyclic voxel buffer in GPU memory,
  • Evaluate and integrate modern methods of Computer Vision into existent pipeline: dvo - dense visual odometry by Tum, libfovis, various algorithms from opencv development and contrib branches
  • Integration of SLAM methods - iSam and g2o
  • Manage, motivate, and build engineering team
  • Participate in non-technical activities ? develop marketing and financing strategy, client development.


Achievements:

  • Cross-platform framework for algorithms evaluation was build. Easy-to-use solution (server side) for 3d-reconstruction was developed providing speed and accuracy of the modern research projects.

C++ OpenCL OpenCV PCL library CUDA Visual odometry methods SLAM algorithms bundle adjustment General Computer vision methods GPGPU computing.
on request
Russia
4 years 2 months
2009-09 - 2013-10

Participate in sales and demo events

Senior Software Engineer - Principal Software Engineer C++ Java Android ...
Senior Software Engineer - Principal Software Engineer

Participate in sales and demo events as technical expert. Contribute to design and implementation of various pilot projects. Improve development processes by introducing the company to the best practices:

  • Setup Continuous Integration build server with control version system, auto-documentation, auto-tests and integration with internal ticketing system (Team City, GNU Make, doxygen, subversion, ant)
  • Develop android application for the needs of Russian Railways (management of railroad tracks)
  • Re-work build scripts for all low-level modules for automated cross-platform builds and testing under Linux\sparc solaris\FreeBSD x86\x64 (cross-compilation, bash scripting for vm with target system orchestration)
  • Develop proof of concepts application for existing modules: SCADA system integrations via SOAP and COM protocols


Achievements:

  • Continuous integration build system was deployed in department of system development. It integrated with repositories of version control system, supported cross-compilation, document generation, and automated testing.

C++ Java Android Understand Doxygen Team City cross-compilation Umbrello
Transset company
1 month
2010-03 - 2010-03

Perform audit of IT services at internet shop

reverse software engineer ida-pro hex-rays scitools understand
reverse software engineer

Perform audit of IT services at internet shop:

  • Re-create the algorithm of cookie generation from isapi dll
  • Disassembling of dll, searching for algorithm's entry point
  • Analyzing and reconstructing call stack
  • Reconstruct algorithm in separate module and using it in order to perform "cookie-hijacking" to access private information


Achievements:

  • 2 critical vulnerabilities and several minor exploits were found. Recommendations for eliminating them were sent to customer.

ida-pro hex-rays scitools understand
private customer
1 year 6 months
2008-04 - 2009-09

Develop and maintain software suite for network devices inventory

software engineer Subversion GNU Make gdb ...
software engineer

Develop and maintain software suite for network devices inventory:

  • Development multi-threaded\multiprocessing Network Management System (NMS) targeting Linux\Solaris x86 & x64 (C\C++, SNMP, pthreads, raw sockets)
  • Sustaining software for polling devices via CORBA protocols (MICO library) according to TMF814 specification (Huawei T2000, ECI Lightsoft, EMS-XDM)
  • Design database structure and stored procedures (Oracle, SQLdeveloper, otl library)
  • Optimize software performance using valgrind with callgrind/cachegrind.
  • Analyzing data flow and proposing new algorithms of data processing as well as enhancing existing ones


Achievements:

  • Speed of device?s discovery were increased by 60 percent. Reliability was increased - amount of bug reports were reduced by more than 10 times.

Subversion GNU Make gdb Source Insight Glow Code Umbrello SQLdeveloper
Transset company
1 year 1 month
2008-04 - 2009-04

Manage of programmer's team (2 programmers)

senior software engineer Firebird IBExpert Borland C++ Builder 6
senior software engineer

Create additional functionality for existing software complex according to dynamically changed requirements:

  • Manage of programmer's team (2 programmers)
  • Analysis and optimization of stored procedures
  • Project infrastructure refactoring
  • Redesign\refactoring source code, DB schemas and various reports and UI forms


Achievements:

  • Under short notice new logic were integrated into existing modules. As a result company (from medical insurance domain) getting approval provide services for the whole region.

Firebird IBExpert Borland C++ Builder 6
MAKS-M company
10 months
2007-06 - 2008-03

Developing CRM application tailored for small to medium typographies

freelance, principal software engineer NSIS Borland C++ Builder 6 WinAPI ...
freelance, principal software engineer

Developing CRM application tailored for small to medium typographies:

  • Requirements gathering and prioritizing
  • Development of system architecture (UML, MS Visio, CRC cards)
  • Database design (MySql and SQLite)
  • Create user?s GUI and interactive help (using JVCL, Globuslib libraries)
  • Manage of programmers team (3 people)
  • Work with OLE (for access to external accounting system - 1C, excel)


Achievements:

  • It were using for three months by our first client - GoodZone typography. Until crisis we had performed several presentations for companies and had got 5 pre-orders and several requests for tuning functionality

NSIS Borland C++ Builder 6 WinAPI Help Scribble Glow Code
1 year 3 months
2006-10 - 2007-12

fix bugs and implement features in main project

Software engineer C\C++ ClearQuality ClearCase ...
Software engineer

Work in sustaining team: fix bugs and implement features in main project (IVR system Nortel MPS 1000, Nuance, Loquendo):

  • issue analysis (based on crash dump or customer logs and report, with description of abnormal behavior of system)
  • investigating reason of error (customer system settings, project configuration, source code)
  • Develop and test solution, provide test cases and other info for patch


Achievements:

  • Setup modules of voice recognition and speech synthesis with Media Processing Server into single deployed system for performing testing (MPS by Nortel, SpeechWorks, Nuance, Loquendo). Several test application were developed for testing (using vxml, peripro).

C\C++ ClearQuality ClearCase GNU Make VC++
Mera-NN
2 years 7 months
2004-04 - 2006-10

Design GUI for easy access to database

Software engineer Firebird IBExpert Borland C++ Builder 6
Software engineer

Design GUI for easy access to database. Has been involved in:

  • Developing in C++ and SQL
  • Technical support
  • Writing the user guide documentation
  • Software testing
  • Analyze of the requirements specification


Achievements:

  • To speed up workflow were developed and deployed several utilities for formatting reports - this work was done manually till that. Synchronization and exchange between accounting database and firebird's database were fully automated - it helps to decrease quantity of invoices-duplicates more than twice.

Firebird IBExpert Borland C++ Builder 6
Maks-M

Aus- und Weiterbildung

Aus- und Weiterbildung

2001 - 2008:

Computational Mathematics and Cybernetics

BS CS

N. I. Lobachevsky State University of Nizhny Novgorod Faculty


2009:

System Programming (Linux Kernel)

Auriga


2011:

Introduction to Artificial Intelligence

Udacity


2013:

Machine Learning

Coursera


2013:

Game Theory

Coursera


2014:

Heterogeneous Parallel Programming

Coursera


2017:

Functional Programming Principles in Scala

Coursera


2017:

Cryptography

Coursera

Kompetenzen

Kompetenzen

Top-Skills

Python Apache Spark PostgreSQL Scala C++ Elastic Search Apache Cassandra Redis Kafka AWS Terraform Kubernetes Helm CI/CD Data-Mart Datawarehouse Applikationsarchitektur

Produkte / Standards / Erfahrungen / Methoden

Key Skills

Libraries/Frameworks:

  • FastApi
  • pandas
  • Django
  • Flask
  • Akka
  • play
  • scikit


Development tools:

  • GCC
  • GNU Make
  • cmake
  • sbt
  • ant
  • maven
  • vim
  • eclipse


Big data ecosystem

  • Spark
  • Kafka
  • Cassandra
  • Impala
  • Hadoop
  • Airflow
  • Hive
  • Impala
  • Hadoop
  • GCD
  • AWS
  • Kubernetes
  • Elasticsearch


Technologies:

  • UML
  • OOP
  • heterogeneous computing
  • multithreading
  • GPGPU programming
  • cross-platform programming
  • SOAP
  • CORBA
  • RPC
  • SOA
  • RESTful api
  • distributed systems
  • TDD


Debugging tools:

  • Gdb
  • dbx
  • IDA Pro
  • hex-rays
  • valgrind
  • trace tools


Version Control Systems:

  • Git
  • Subversion
  • Clearcase


Bug/Request tracking systems:

  • ClearQuality
  • Redmine
  • Jira
  • Gitlab


Continuous integration tools:

  • Team City
  • Jenkins

Continuous deployment tools:
  • Terraform
  • Github actions
  • docker


Other tools/frameworks:

  • Scitools understand
  • Source Insight
  • wireshark
  • tcpdump
  • Crucible
  • grunt
  • gulp
  • upsource


Summary

  • 20 years within industry, for more than 9 years I have been in charge of designing the architecture of complex systems that can easily scale by choosing tech stack, define core components and implement performance critical parts. I have been hiring, coaching and managing a team of developers as well as run integration within third party agencies.
  • Deep understanding of OS internals in addition to a wide exposure to modern technologies allow me to quickly prototype proof-of-concept solutions to showcase different approaches to address demands of clients.
  • Math background and passion for advanced concepts of Computer Science (AI, ML) allow me to effectively integrate innovative approaches of data processing and analysis into software of various domains (oil refining, telecom, advertisement).
  • Ability to lead and coach a team by bringing values of hacker culture grant me the acquaintance of many high caliber professionals who can be used for any time-critical projects as a freelancers/consultants to meet strict deadlines.


Professional interests

  • Software and team architecture
  • design of complex systems
  • computer vision
  • heterogeneous computing
  • algorithm parallelization
  • application of artificial intelligence
  • GPGPU programming
  • functional programming

Programmiersprachen

Python/Flutter/SQL/C++11
Java
Scala
shell scripting

Datenbanken

Postgres
Sqlite
Cassandra
Redis
Oracle
Aerospike

Einsatzorte

Einsatzorte

Deutschland, Schweiz, Österreich
möglich

Projekte

Projekte

2024-10 - now

prepare system to scale, Performance tuning

principal engineer MongoDB Capacity Management Performance-Management ...
principal engineer

Architecture review of code C# and schema for MongoDB.

Prepare recommendation on schema design to scale from 7 billions rows to 14 billions: partition data, choosing hash functions to avoid collisions, introduce cache and queue to scale processing, address security implications for PII data salting, encryption, data anonimisation.


MongoDB Capacity Management Performance-Management Performance Measurement Stresstest
2 months
2024-08 - 2024-09

Audit of software development processes, decrease time-to-market

Principal engineer Changemanagement Continuous Delivery Continuous Integration ...
Principal engineer

Review and analyse existing development practices and processes

identify inefficiencies and bottlenecks

prepare a roadmap to improve time-to-market: optimise CI/CD processes, introduce autotests, perform vertical task slicing to facilitate working in small batches, prepare RFC/ADR before hand to align team on software design

Changemanagement Continuous Delivery Continuous Integration Testautomatisierung DevOps Azure Devops C#
CMO Group
2 months
2024-06 - 2024-07

Performance tuning - 2k read/writes per second - postgres, Django, python

PostgreSQL Django Python ...

Analyse Postgres data access patterns: read vs writes, hot tables, redundant and missing indexes, table and index sizes.

Check analyze and vacuum stats.

Collect query statistics - number of calls, mean and max time.

Review app architecture - job's queue, webhook and user triggered endpoint + kronjobs in kubernetes.

Collect metrics in prometheus and grafana related to db usage and code execution to map CPU spikes with suspicious code.

Together with engineering team run several sessions of code optimisation's that allow to improve overall performance 3 times.

Document runbooks and roadmap to prepare codebase for further growth.  

PostgreSQL Django Python Redis
Rakt technologies
2 years 6 months
2022-01 - 2024-06

Build a Uber-like match-making platform for cleaners and home-owners

CTO PostgreSQL Python Elastic Search ...
CTO

Tech co-founder:
  • Scale cross functional team from 0 to 12, during last 3 years - zero employee's churn
  • Decrease average time to market from week to 2 days
  • Small batches, automate everything, product mentality, extreme ownership, mob-programming and KISS principles, Agile (Scrum/Kanban), Kaizen, TDD,
  • Define and help to execute tech strategy to meet business needs - budgeting, risks vs priorities, tech stack and architecture, build vs buy


Create CleanBee - Uber-like platform to connect homeowners with cleaners:

  • Build initial version of backend Rest API (Python, FastAPI, pydantic, alembic, sqlalchemy, asyncio, mypy, pytest, psycopg, firebase, pre-commit, ruff)
  • Build initial version of SPA - web frontend (NextJS, javascript, typescript, tailwind, css)
  • Develop match-making search/recommendation engine and wrap it behind Rest API - feature engineering, core engine - KNN + ALS (scikit, redis, numpy, pandas)
  • Design data pipelines to analyze and create dashboards for conversion funnels (Elasticsearch, Google Query, PostgreSQL, Firebase analytics, data governance, data cleansing, data quality)
  • Setup CI/CD and monitoring (Github actions, terraform, k8s, helm, vault, grafana, prometheus, ELK, GCD, AWS, docker, PostgreSQL, fastlane)
  • Integration of social signing, analytics, support chat and payment into mobile app (MVVM, flutter, clean architecture, getstream, firebase notification deeplinks)
  • UI & integration tests (curated env with auto-generated dummy data + testproject, apium, patrol, firebase, test project)


Create MVP app (iOS & Android) to control facial expressions - on device computer vision pipelines:

  • Port C++ core engine from x86_64 to arm64 (cross-compilation, cross-toolchain)
  • Integrate C++ into native code (jni, CMake, ninja, bazel)
  • Convert tensorflow model to tensorflow lite format (knowledge distillation, quantisation, prunning)
  • To speedup prototyping and algorithm assessment - create an demo pipeline to experiment with various model?s architecture and pre/post processing (python, javascript, react, flask, tensorflow, tensorboard, CVAT, DVC) 

AWS Google Cloud Azure
PostgreSQL Python Elastic Search AWS GCD Azure Hive AWS: EC2 EMR ECS S3 K8S Terraform c++ Kotlin Swift javascript tensorflow Docker github actions for CI\CD AWS Sagemaker Mechanical Turk DVC
Syzygy AI LLC
Berlin
2 years 8 months
2019-06 - 2022-01

Data infra cloud migration

senior data engineer Apache Spark Scala Python ...
senior data engineer
First senior engineer in data department - help company transition towards Data Mesh topology:
  • Migrate number of data pipelines from Cloudera private hosting environment into AWS cloud to support exponential demand for growth during Covid
  • Develop framework to assure data quality of digested events
  • Develop new data pipelines and extend existing to fulfill stackholder?s needs
  • Participate in on-call rotation to support SLA for business critical datasets
  • Performance tuning of data pipelines: code of ETLs, Spark & HDFS internals
AWS
Apache Spark Scala Python AWS Terraform Kubernetes Hive Impala Hadoop Spark stream processing Airflow Parquet AWS: EC2 EMR S3 DMS Sage Maker K8S
Hellofresh
Berlin
1 year 1 month
2018-06 - 2019-06

Manage development team of 3 developers

Co-founder & principal software engineer microservices application containerisation Python ...
Co-founder & principal software engineer
Manage development team of 3 developers:
  • Manage priorities and define project?s milestones
  • Gather and convert functional requirements to sprint?s tasks
  • Run code review and pair-coding sessions
  • Provide mentorship and assistance during criticals


Create automated crypto currency-trading system:

  • Develop scalable trading platform - high throughput low-latency fully automated trading engine for multi-agent deployments
  • Design reactive architecture to unify integration of additional exchanges
  • Lead implementation of unified UI for stakeholders:
  • rules management
  • deployment automation and monitoring
  • trend exploration and data discovery
  • Create dedicated services for live data stream acquisition and real time signaling
  • Inference engine for models backtesting
microservices application containerisation Python gevent SOA redis based event queues aws stream processing websockets postgres flask telegram bot api
on request
2 years 8 months
2015-11 - 2018-06

Build analytical project portfolio

senior software architect Spark Cassandra Zeppelin ...
senior software architect
Senior Architect:

  • Design, build and optimize high throughput highly available multi component data processing pipelines (up to 80k events per seconds, depending on cluster configuration).
  • Re-factor of business critical components of various products across the whole stack
  • Cassandra, Kafka, Elastic performance tuning - schema adjustments & implementation of high performance multithreaded shooters in C++, jvm & linux kernel optimization
  • Containerization of applications: maintaining internal docker registry, automate orchestration of dockers vms, docker performance tuning
  • Introduce company for modern SDLC practices and help to integrate it in a way to be coherent with business goals: continuous integration, devops, integration tests & auto-tests
  • Participate in business critical activities: sizing and hardware planning, develop POC for pre-sales events, requirements assessments, investigation of disaster incidents, etc
  • Develop and run internship program: software development, data analysis.


Head of analytics department:

  • Add several key functionalities for company's products portfolio.
  • Implement various analytical modules on top of Apache Spark.
  • Build a toolbox for algorithms assessments for data discovery and quick hypothesis checks
  • Assessment and adaptation of methods of AI\ML for such tasks as predictions\clusterization.
  • Application of methods Natural Language Processing (NLP) according to business goals.


Achievements:

  • Decrease storage requirements on 50%. Increase overall system performance from state when the full cluster cant deal with peak load to situation when ? of cluster?s nodes weren?t available due to power outage new system handle load without hiccups. 
  • Develop proof of concept application that lead to signing three new contracts with major clients for the whole company history.

Spark Cassandra Zeppelin postgres redis kafka akka scala Java SOA microservices reverse engineering application of math statistics data mining predictive analysis & DSP C++ boost python scikit docker
Security company
Abu Dhabi
11 months
2015-01 - 2015-11

Define and control execution of sprints for freelancers

Senior software engineer Algorithms machine learning postgres ...
Senior software engineer
Perform duty of CTO (july - september):
  • Manage hiring for backend/frontend/system administrator in order to meet project?s deadline
  • Define and control execution of sprints for freelancers
  • Provide technical leadership and coaching for team


Architect a data mining sub-system. Design, build and optimize search/recommendation engine:

  • Develop general architecture of server components, choose technology stack for
  • implementation, design database structure and implement middleware layer for interaction with 3rd party systems
  • Build web-crawler for machine learning algorithms evaluation
  • Perform evaluation of various machine learning techniques, embed most promising approach to general pipeline
  • Adapt core of recommendation engine for high-load project by profile and optimize
  • bottlenecks
  • Define and manage Mesos cluster deployment strategy
  • Control and manage of integration various clients (mobile/web) to backend services


Achievements:

  • Accuracy of search above 85 percent, time of reply less than 250 ms

Algorithms machine learning postgres elasticsearch redis scikit scrapy nltk gensim selenium Mesos cluster stack AWS stack vagrant docker git django SOAP RESTfull api
Insydo
Dubai
2 years 8 months
2012-05 - 2014-12

Develop application for 3d-scene reconstruction from RGB-D cameras

CTO/founder C++ OpenCL OpenCV ...
CTO/founder
Develop application for 3d-scene reconstruction from RGB-D cameras:
  • Porting reference kinect fusion implementation from CUDA, PCL library to OpenCL
  • Collaborate with PCL community to contribute to KinFu sub-module in order to overcome limitations of existing implementation: integrate RGB information to resulting point cloud, cyclic voxel buffer in GPU memory,
  • Evaluate and integrate modern methods of Computer Vision into existent pipeline: dvo - dense visual odometry by Tum, libfovis, various algorithms from opencv development and contrib branches
  • Integration of SLAM methods - iSam and g2o
  • Manage, motivate, and build engineering team
  • Participate in non-technical activities ? develop marketing and financing strategy, client development.


Achievements:

  • Cross-platform framework for algorithms evaluation was build. Easy-to-use solution (server side) for 3d-reconstruction was developed providing speed and accuracy of the modern research projects.

C++ OpenCL OpenCV PCL library CUDA Visual odometry methods SLAM algorithms bundle adjustment General Computer vision methods GPGPU computing.
on request
Russia
4 years 2 months
2009-09 - 2013-10

Participate in sales and demo events

Senior Software Engineer - Principal Software Engineer C++ Java Android ...
Senior Software Engineer - Principal Software Engineer

Participate in sales and demo events as technical expert. Contribute to design and implementation of various pilot projects. Improve development processes by introducing the company to the best practices:

  • Setup Continuous Integration build server with control version system, auto-documentation, auto-tests and integration with internal ticketing system (Team City, GNU Make, doxygen, subversion, ant)
  • Develop android application for the needs of Russian Railways (management of railroad tracks)
  • Re-work build scripts for all low-level modules for automated cross-platform builds and testing under Linux\sparc solaris\FreeBSD x86\x64 (cross-compilation, bash scripting for vm with target system orchestration)
  • Develop proof of concepts application for existing modules: SCADA system integrations via SOAP and COM protocols


Achievements:

  • Continuous integration build system was deployed in department of system development. It integrated with repositories of version control system, supported cross-compilation, document generation, and automated testing.

C++ Java Android Understand Doxygen Team City cross-compilation Umbrello
Transset company
1 month
2010-03 - 2010-03

Perform audit of IT services at internet shop

reverse software engineer ida-pro hex-rays scitools understand
reverse software engineer

Perform audit of IT services at internet shop:

  • Re-create the algorithm of cookie generation from isapi dll
  • Disassembling of dll, searching for algorithm's entry point
  • Analyzing and reconstructing call stack
  • Reconstruct algorithm in separate module and using it in order to perform "cookie-hijacking" to access private information


Achievements:

  • 2 critical vulnerabilities and several minor exploits were found. Recommendations for eliminating them were sent to customer.

ida-pro hex-rays scitools understand
private customer
1 year 6 months
2008-04 - 2009-09

Develop and maintain software suite for network devices inventory

software engineer Subversion GNU Make gdb ...
software engineer

Develop and maintain software suite for network devices inventory:

  • Development multi-threaded\multiprocessing Network Management System (NMS) targeting Linux\Solaris x86 & x64 (C\C++, SNMP, pthreads, raw sockets)
  • Sustaining software for polling devices via CORBA protocols (MICO library) according to TMF814 specification (Huawei T2000, ECI Lightsoft, EMS-XDM)
  • Design database structure and stored procedures (Oracle, SQLdeveloper, otl library)
  • Optimize software performance using valgrind with callgrind/cachegrind.
  • Analyzing data flow and proposing new algorithms of data processing as well as enhancing existing ones


Achievements:

  • Speed of device?s discovery were increased by 60 percent. Reliability was increased - amount of bug reports were reduced by more than 10 times.

Subversion GNU Make gdb Source Insight Glow Code Umbrello SQLdeveloper
Transset company
1 year 1 month
2008-04 - 2009-04

Manage of programmer's team (2 programmers)

senior software engineer Firebird IBExpert Borland C++ Builder 6
senior software engineer

Create additional functionality for existing software complex according to dynamically changed requirements:

  • Manage of programmer's team (2 programmers)
  • Analysis and optimization of stored procedures
  • Project infrastructure refactoring
  • Redesign\refactoring source code, DB schemas and various reports and UI forms


Achievements:

  • Under short notice new logic were integrated into existing modules. As a result company (from medical insurance domain) getting approval provide services for the whole region.

Firebird IBExpert Borland C++ Builder 6
MAKS-M company
10 months
2007-06 - 2008-03

Developing CRM application tailored for small to medium typographies

freelance, principal software engineer NSIS Borland C++ Builder 6 WinAPI ...
freelance, principal software engineer

Developing CRM application tailored for small to medium typographies:

  • Requirements gathering and prioritizing
  • Development of system architecture (UML, MS Visio, CRC cards)
  • Database design (MySql and SQLite)
  • Create user?s GUI and interactive help (using JVCL, Globuslib libraries)
  • Manage of programmers team (3 people)
  • Work with OLE (for access to external accounting system - 1C, excel)


Achievements:

  • It were using for three months by our first client - GoodZone typography. Until crisis we had performed several presentations for companies and had got 5 pre-orders and several requests for tuning functionality

NSIS Borland C++ Builder 6 WinAPI Help Scribble Glow Code
1 year 3 months
2006-10 - 2007-12

fix bugs and implement features in main project

Software engineer C\C++ ClearQuality ClearCase ...
Software engineer

Work in sustaining team: fix bugs and implement features in main project (IVR system Nortel MPS 1000, Nuance, Loquendo):

  • issue analysis (based on crash dump or customer logs and report, with description of abnormal behavior of system)
  • investigating reason of error (customer system settings, project configuration, source code)
  • Develop and test solution, provide test cases and other info for patch


Achievements:

  • Setup modules of voice recognition and speech synthesis with Media Processing Server into single deployed system for performing testing (MPS by Nortel, SpeechWorks, Nuance, Loquendo). Several test application were developed for testing (using vxml, peripro).

C\C++ ClearQuality ClearCase GNU Make VC++
Mera-NN
2 years 7 months
2004-04 - 2006-10

Design GUI for easy access to database

Software engineer Firebird IBExpert Borland C++ Builder 6
Software engineer

Design GUI for easy access to database. Has been involved in:

  • Developing in C++ and SQL
  • Technical support
  • Writing the user guide documentation
  • Software testing
  • Analyze of the requirements specification


Achievements:

  • To speed up workflow were developed and deployed several utilities for formatting reports - this work was done manually till that. Synchronization and exchange between accounting database and firebird's database were fully automated - it helps to decrease quantity of invoices-duplicates more than twice.

Firebird IBExpert Borland C++ Builder 6
Maks-M

Aus- und Weiterbildung

Aus- und Weiterbildung

2001 - 2008:

Computational Mathematics and Cybernetics

BS CS

N. I. Lobachevsky State University of Nizhny Novgorod Faculty


2009:

System Programming (Linux Kernel)

Auriga


2011:

Introduction to Artificial Intelligence

Udacity


2013:

Machine Learning

Coursera


2013:

Game Theory

Coursera


2014:

Heterogeneous Parallel Programming

Coursera


2017:

Functional Programming Principles in Scala

Coursera


2017:

Cryptography

Coursera

Kompetenzen

Kompetenzen

Top-Skills

Python Apache Spark PostgreSQL Scala C++ Elastic Search Apache Cassandra Redis Kafka AWS Terraform Kubernetes Helm CI/CD Data-Mart Datawarehouse Applikationsarchitektur

Produkte / Standards / Erfahrungen / Methoden

Key Skills

Libraries/Frameworks:

  • FastApi
  • pandas
  • Django
  • Flask
  • Akka
  • play
  • scikit


Development tools:

  • GCC
  • GNU Make
  • cmake
  • sbt
  • ant
  • maven
  • vim
  • eclipse


Big data ecosystem

  • Spark
  • Kafka
  • Cassandra
  • Impala
  • Hadoop
  • Airflow
  • Hive
  • Impala
  • Hadoop
  • GCD
  • AWS
  • Kubernetes
  • Elasticsearch


Technologies:

  • UML
  • OOP
  • heterogeneous computing
  • multithreading
  • GPGPU programming
  • cross-platform programming
  • SOAP
  • CORBA
  • RPC
  • SOA
  • RESTful api
  • distributed systems
  • TDD


Debugging tools:

  • Gdb
  • dbx
  • IDA Pro
  • hex-rays
  • valgrind
  • trace tools


Version Control Systems:

  • Git
  • Subversion
  • Clearcase


Bug/Request tracking systems:

  • ClearQuality
  • Redmine
  • Jira
  • Gitlab


Continuous integration tools:

  • Team City
  • Jenkins

Continuous deployment tools:
  • Terraform
  • Github actions
  • docker


Other tools/frameworks:

  • Scitools understand
  • Source Insight
  • wireshark
  • tcpdump
  • Crucible
  • grunt
  • gulp
  • upsource


Summary

  • 20 years within industry, for more than 9 years I have been in charge of designing the architecture of complex systems that can easily scale by choosing tech stack, define core components and implement performance critical parts. I have been hiring, coaching and managing a team of developers as well as run integration within third party agencies.
  • Deep understanding of OS internals in addition to a wide exposure to modern technologies allow me to quickly prototype proof-of-concept solutions to showcase different approaches to address demands of clients.
  • Math background and passion for advanced concepts of Computer Science (AI, ML) allow me to effectively integrate innovative approaches of data processing and analysis into software of various domains (oil refining, telecom, advertisement).
  • Ability to lead and coach a team by bringing values of hacker culture grant me the acquaintance of many high caliber professionals who can be used for any time-critical projects as a freelancers/consultants to meet strict deadlines.


Professional interests

  • Software and team architecture
  • design of complex systems
  • computer vision
  • heterogeneous computing
  • algorithm parallelization
  • application of artificial intelligence
  • GPGPU programming
  • functional programming

Programmiersprachen

Python/Flutter/SQL/C++11
Java
Scala
shell scripting

Datenbanken

Postgres
Sqlite
Cassandra
Redis
Oracle
Aerospike

Vertrauen Sie auf Randstad

Im Bereich Freelancing
Im Bereich Arbeitnehmerüberlassung / Personalvermittlung

Fragen?

Rufen Sie uns an +49 89 500316-300 oder schreiben Sie uns:

Das Freelancer-Portal

Direktester geht's nicht! Ganz einfach Freelancer finden und direkt Kontakt aufnehmen.