Introduction - FullStack Developer/ Data Engineer

Thank you for the briefing. Now, I can understand more about the position. After listening to the conversation, I am more confident to say that, “I am best fit for this role.”

Introduction

I am a Full-Stack Python Developer, with all years of experience, with around 80% into Backend work and 20% into the frontend.

Backend Expertise

In Backend, I worked on:

• Creating standalone scripts for automation, scheduled jobs, ETL jobs, and data pipelines.
• Building RESTful APIs and web applications.

Frontend Expertise

In Frontend, I worked on:

• Creating dashboards, graphs, and charts using d3.js or Highcharts libraries.
• Building tables using ag-grid.
• Working with JavaScript, jQuery, and React.

Databases

In databases:

• Relational Databases: MySQL, MS SQL, Oracle DB, PostgreSQL.
• NoSQL Databases: MongoDB, Cassandra.
• Data Warehousing: Snowflake schema.

Python Expertise

In Python:

• Creating web applications and RESTful APIs using Django, Flask, and FastAPI frameworks.
• Process automation and integrating with infrastructure (Linux/Windows).
• Data gathering from:
  1. Structured datasources like REST APIs (internal or external), databases, etc.
  2. Unstructured datasources like web scraping using Beautiful Soup.
  3. Structured, semi-structured, or unstructured file types like CSV, Excel, JSON, YAML, Parquet, etc.
• Following TDD (Test-Driven Development) by creating unit tests and integration tests using unittest or pytest modules.

Caching and Scheduling

In caching:

• Worked with Redis and Memcache.

In scheduling:

• Worked with Celery.
• Integrated Celery with Django applications for periodic jobs.

For data job orchestration, I worked with Airflow.

Public Cloud Experience

In Public Cloud, I am mostly associated with AWS Cloud. In AWS Cloud, I worked with:

Server-Based Architectures

• EC2 Instances
• Elastic Beanstalk
• Elastic Load Balancer
• Auto-Scaling
• Route53

Storage Solutions

• S3 Bucket for file storage
• S3 Glacier for archival storage

Databases

• AWS RDS & Redshift for relational databases
• AWS DynamoDB for NoSQL

Serverless Architectures

• AWS Lambda (time-triggered or event-triggered)
• AWS API Gateway (HTTP-triggered)
• AWS Event Scheduler for scheduling Lambda

Container-based Environments

• Created docker.yaml files for creating containers
• AWS EKS (Elastic Kuberntes Service) for Orchestrating the pods.
• Also, wrote the helm chart

For OAuth, I worked with AWS Cognito.

Also worked with SQS, SNS, and SES.

For big data processing, I worked with EMR clusters for PySpark.

For ETLs, I worked with AWS Glue Jobs with DataCatalog and PySpark.

CI/CD and Infrastructure as Code

Experience in creating CI/CD setups using Jenkins and Groovy scripts.

In terms of Infrastructure as Code, I worked mainly with:

• CloudFormation templates
• Terraform
• Pulumi Python module

Agile Methodologies

Experience in agile methodologies like Scrum and Kanban in facilitating agile ceremonies like:

• Daily standups
• Sprint reviews
• Planning sessions
• Retrospectives

Familiar with Agile tools like Jira and skilled in working with cross-functional teams including Dev teams, QA testers, and PM/POs.