About Me
I am a Senior Software Engineer at Astronomer, where I work on the design and engineering of Apache Airflow - an open source data orchestration platform. Astronomer is the orchestration platform for the modern data stack. Built on Kubernetes, Astronomer makes it easy to run, monitor, and scale Apache Airflow clusters in our cloud or yours. I’m passionate about building scalable data infrastructure and contributing to open source projects that impact the developer community.
As a strong advocate of open source and believer in “Learning by Doing,” I have contributed to several high-impact projects including Apache Airflow, Ask-Astro, Universal Transfer Operator, Astro-Python-SDK, Astronomer-Providers, and OpenCV CVAT.
📋 Table of Contents
Experience
1
Jan 2022 — Present
Senior Software Engineer
Astronomer
• Spearheaded key features in Apache Airflow 3.0 release, pivotal to Astronomer's $93M Series D momentum
• AIP-83: Independently designed and developed the proposal enabling Airflow to directly support model training and generative AI inference without hacks or workarounds
• Architected and built ruff linter and configuration tools enabling Airflow users worldwide to seamlessly upgrade from 2.x to 3
• Contributed to AIP-72, migrating Airflow to a client–server architecture for multi-cloud deployments and multi-language support
• Led development of Astro Runtime (Astronomer's production-ready Airflow distribution that bundles Airflow, core providers, and dependencies into standardized, secure Docker images)
• Designed and engineered the `astronomer-providers-logging` module to handle task logs in Airflow via HTTP endpoints, reducing the Astro Runtime Docker image size by 80% by removing heavy providers
• Managed 80% of Astro Runtime releases with complex dependencies on Apache Airflow, providing Day Zero support for Astronomer customers
• Pioneered nightly build releases to catch bugs early in Airflow/Astro, reducing manual regression testing by 90%
• Designed and built Ask-Astro — an open-source chatbot for Apache Airflow/Astronomer Q&A, using Airflow to orchestrate data ingestion, embed content into a vector database, and incorporate a feedback loop (source code)
• Architected and implemented apache-airflow-providers-transfers for Airflow, providing a single agnostic interface to move data between sources and destinations, replacing the need for 100+ specialised transfer operators
• Co-designed and developed the Astro Python SDK for rapid ETL development in Airflow, abstracting away DAG boilerplate so engineers can focus on business logic
• Implemented deferrable mode for 80+ Airflow operators/sensors, freeing worker slots while waiting on external events and reducing compute costs by up to 70%
• Delivered presentations at Apache Airflow Summit (2022—2025) and keynote at Microsoft IDC Hyderabad on advanced Airflow topics
• Contributed extensively to Apache Airflow (core and providers) with new features, bug fixes, and documentation improvements
• Established guidelines and guardrails for using generative AI coding tools (Cursor, GitHub Copilot) to boost developer productivity
• AIP-83: Independently designed and developed the proposal enabling Airflow to directly support model training and generative AI inference without hacks or workarounds
• Architected and built ruff linter and configuration tools enabling Airflow users worldwide to seamlessly upgrade from 2.x to 3
• Contributed to AIP-72, migrating Airflow to a client–server architecture for multi-cloud deployments and multi-language support
• Led development of Astro Runtime (Astronomer's production-ready Airflow distribution that bundles Airflow, core providers, and dependencies into standardized, secure Docker images)
• Designed and engineered the `astronomer-providers-logging` module to handle task logs in Airflow via HTTP endpoints, reducing the Astro Runtime Docker image size by 80% by removing heavy providers
• Managed 80% of Astro Runtime releases with complex dependencies on Apache Airflow, providing Day Zero support for Astronomer customers
• Pioneered nightly build releases to catch bugs early in Airflow/Astro, reducing manual regression testing by 90%
• Designed and built Ask-Astro — an open-source chatbot for Apache Airflow/Astronomer Q&A, using Airflow to orchestrate data ingestion, embed content into a vector database, and incorporate a feedback loop (source code)
• Architected and implemented apache-airflow-providers-transfers for Airflow, providing a single agnostic interface to move data between sources and destinations, replacing the need for 100+ specialised transfer operators
• Co-designed and developed the Astro Python SDK for rapid ETL development in Airflow, abstracting away DAG boilerplate so engineers can focus on business logic
• Implemented deferrable mode for 80+ Airflow operators/sensors, freeing worker slots while waiting on external events and reducing compute costs by up to 70%
• Delivered presentations at Apache Airflow Summit (2022—2025) and keynote at Microsoft IDC Hyderabad on advanced Airflow topics
• Contributed extensively to Apache Airflow (core and providers) with new features, bug fixes, and documentation improvements
• Established guidelines and guardrails for using generative AI coding tools (Cursor, GitHub Copilot) to boost developer productivity
Python
Apache Airflow
LLM
LangChain
AWS
GCP
Docker
Ruff
2
Jan 2019 — Dec 2021
Senior Software Engineer
Wadhwani AI
• Hired and led engineering teams across multiple projects in healthcare and agriculture domains
• Integrated pest management solution:
• Designed, developed and deployed a pest management system which runs a PyTorch-based model on the phone/cloud to identify harmful pests, alert farmers about infestation and reduce crop loss in cotton farming. Currently being used by over 11,000 farmers
• Developed data pipeline for data collection using ODK, annotation using LabelMe, model deployment (online on-cloud and offline on-device), analytics dashboard along with open APIs for third-party integration and Django backend
• Part of the team that won a $2 million grant as part of the Google AI Impact Challenge offered to only 20 teams worldwide, becoming the only team from India to do so and featuring in Google I/O and Google for India
• Led engineering effort for annotation studio which is data (artifacts and annotation) management and job orchestration platform to manage, schedule and orchestrate annotation jobs across multiple annotation vendors using Apache Airflow, FastAPI and React. The annotations are used for model training and performance decisions
• Reduced the video buffering time by 75% on open-source CVAT annotation tool by adding Redis cache to cache the frames during data ingestion
• Engineered various microservices being used by multiple teams with over 0.5M events to date like authentication and authorization service, geospatial directory and emailing-service using Django, AWS SES, OAuth 2.0, AWS Cognito and Fargate
• Built a COVID-19 mobility dashboard integrating with COVID-19 Mobility Data Network and Facebook Data For Good which enabled 11 state government authorities in India with actionable insights to optimize emergency protocols in line with epidemic spread and identifying high movement areas
• Integrated pest management solution:
• Designed, developed and deployed a pest management system which runs a PyTorch-based model on the phone/cloud to identify harmful pests, alert farmers about infestation and reduce crop loss in cotton farming. Currently being used by over 11,000 farmers
• Developed data pipeline for data collection using ODK, annotation using LabelMe, model deployment (online on-cloud and offline on-device), analytics dashboard along with open APIs for third-party integration and Django backend
• Part of the team that won a $2 million grant as part of the Google AI Impact Challenge offered to only 20 teams worldwide, becoming the only team from India to do so and featuring in Google I/O and Google for India
• Led engineering effort for annotation studio which is data (artifacts and annotation) management and job orchestration platform to manage, schedule and orchestrate annotation jobs across multiple annotation vendors using Apache Airflow, FastAPI and React. The annotations are used for model training and performance decisions
• Reduced the video buffering time by 75% on open-source CVAT annotation tool by adding Redis cache to cache the frames during data ingestion
• Engineered various microservices being used by multiple teams with over 0.5M events to date like authentication and authorization service, geospatial directory and emailing-service using Django, AWS SES, OAuth 2.0, AWS Cognito and Fargate
• Built a COVID-19 mobility dashboard integrating with COVID-19 Mobility Data Network and Facebook Data For Good which enabled 11 state government authorities in India with actionable insights to optimize emergency protocols in line with epidemic spread and identifying high movement areas
Python
Django
FastAPI
React
Apache Airflow
PyTorch
AWS
Redis
OAuth 2.0
3
Dec 2017 — Jan 2019
Software Development Engineer
DreamVu Inc.
• Developed end-to-end modular video acquisition and processing software for VR/AR and machine vision applications (for in-house camera) along with SDK for third party integration which helped to secure an angel funding of $0.5 million from SRI capital
Computer Vision
VR/AR
SDK Development
Camera Calibration
3D Video
4
Aug 2016 — Nov 2017
Software Engineer
AapkaPainter
• Built CRM, task management, and sales automation systems; integrated Razorpay payment gateway processing 10M+ INR monthly; developed Flask API for automated floor plan measurement extraction using computer vision
CodeIgniter
Flask
Payment Gateway
Computer Vision
CRM
5
Mar 2015 — Jun 2015
Software Intern
Almabase
• Built scheduled ride booking web app with notifications; developed web crawler for alumni news extraction using BeautifulSoup and regex; implemented OpenCV-based face detection system to extract alumni photos from archived documents
Web Development
BeautifulSoup
OpenCV
Web Scraping
📋 View detailed experience history on LinkedIn →
Skills & Expertise
Core Concepts: Data Structures and Algorithms, Object-Oriented Programming, System Design, Data Engineering, LLMOps, Large Language Models (LLM), Conversational AI, Geo-spatial Data, REST APIs, CI/CD, Release Management
Programming Languages: Python (Proficient), C++ (Intermediate), JavaScript (Intermediate), PHP (Intermediate), C (Prior Experience), Java (Prior Experience), Bash, LaTeX
Frameworks and Libraries: Apache Airflow, Django, FastAPI, Flask, LangChain, OpenCV, OpenLineage, Apache Livy
Databases: PostgreSQL, MySQL, MongoDB, Weaviate, SQL
Cloud and Infrastructure: AWS, Google Compute Engine, Azure, Docker, OAuth 2.0
DevOps and Tools: Git, GitHub Actions, CircleCI, Swagger, Postman
Leadership: Team Management, Technical Hiring, Mentoring, Stakeholder Management, Technical Strategy, Architecture Reviews, Process Improvement, Vendor Management, Cross-functional Collaboration, SDLC, Agile, Sprint Planning
Open Source Contributions
- apache-airflow - Core Apache Airflow platform contributions
- ask-astro - Open-source chatbot for Apache Airflow/Astronomer Q&A
- ruff - Python linter contributions
- apache-airflow-provider-transfers - Universal Transfer Operator
- astro-sdk - Python SDK for rapid ETL development in Airflow
- astronomer-providers - Apache Airflow providers with deferrable operators
Education
1
2012 — 2016
Bachelor's in Technology, Computer Science and Engineering
National Institute of Technology Warangal
• Grade: 6.44/10
• Coordinator at Technozion 2015 - South India's second largest technical fest
• General Secretary at Electronics, Amateur and HAM Club NIT Warangal
• National Robotics Championship (Robo-war) winner
• Member of Innovation Garage (IG) 2014-2016
• Founded startup that built real-time position reporting system for buses without GPS
• Worked on AR/VR camera applications, drone data analysis, and ML-based web solutions
• Fellow at Teach For India contributing to education and social impact
• Coordinator at Technozion 2015 - South India's second largest technical fest
• General Secretary at Electronics, Amateur and HAM Club NIT Warangal
• National Robotics Championship (Robo-war) winner
• Member of Innovation Garage (IG) 2014-2016
• Founded startup that built real-time position reporting system for buses without GPS
• Worked on AR/VR camera applications, drone data analysis, and ML-based web solutions
• Fellow at Teach For India contributing to education and social impact
Computer Science
Robotics
Leadership
Innovation
Startup
2
2014 — 2016
Bachelor's degree, Technology Entrepreneurship Program
Indian School of Business Hyderabad
• Grade: 81.2%
• Won Design Challenge by ISB Hyderabad
• Participated in technology entrepreneurship programs focusing on startup development, business model creation, and product management
• Founded award-winning startup that built real-time position reporting system for buses without GPS
• Pitched idea to Telangana IT minister and received seed funding offer from TSRTC
• Gained hands-on experience in building scalable technology solutions and understanding market needs
• Won Design Challenge by ISB Hyderabad
• Participated in technology entrepreneurship programs focusing on startup development, business model creation, and product management
• Founded award-winning startup that built real-time position reporting system for buses without GPS
• Pitched idea to Telangana IT minister and received seed funding offer from TSRTC
• Gained hands-on experience in building scalable technology solutions and understanding market needs
Entrepreneurship
Product Management
Business Strategy
Startup
Innovation
3
2009 — 2011
Grade 12
SOS Hermann Gmeiner Higher Secondary School Bharatpur, Nepal
• Head boy of the school
• Active in Robotics activities and competitions
• Head Designer of School Magazine
• Member of Debate team
• Youth Red Cross member
• Scout participant
• Guitar player
• Eco-club member
• Developed strong foundation in analytical thinking and problem-solving skills that later contributed to success in computer science and engineering
• Active in Robotics activities and competitions
• Head Designer of School Magazine
• Member of Debate team
• Youth Red Cross member
• Scout participant
• Guitar player
• Eco-club member
• Developed strong foundation in analytical thinking and problem-solving skills that later contributed to success in computer science and engineering
Leadership
Robotics
Design
Debate
Community Service
Certifications
- Astronomer Certification - DAG Authoring for Apache Airflow (Jan 2022)
- Astronomer Certification - Apache Airflow Fundamentals (Jan 2022)
- Apache Airflow - Udemy Certification (Jan 2021)
- SQL - HackerRank Certification (Sep 2020)
Talks & Publications
I’ve delivered technical talks at major conferences including Apache Airflow Summit (2022-2025), Microsoft IDC Hyderabad, and published research at KDD 2020.
Recent Highlights:
- 2025: Two talks at Apache Airflow Summit Seattle (Beyond Execution Dates, Seamless Airflow Upgrades)
- 2024: Advanced Dataset Scheduling talk at Airflow Summit San Francisco
- 2023: Data pipeline patterns from ETL to ML to LLM at Microsoft IDC Hyderabad
- 2022: Deferrable Operators talk at Airflow Summit 2022
- 2020: Published research paper on AI-powered pest management at KDD 2020
📚 View all talks, presentations, and publications → {#talks-link}
Notable Projects
Ask-Astro
LLM-powered chatbot for Apache Airflow community with RAG and Weaviate vector store. Integrates Apache Airflow with advanced LLM technology to create a versatile chatbot and knowledge interface, leveraging Retrieval Augmented Generation for enhanced AI-driven query response capabilities.
Python
Apache Airflow
LLM
RAG
Weaviate
Universal Transfer Operator
Unified interface that simplifies data transfer workflows in Apache Airflow. Offers a consistent agnostic interface, improving user experience by reducing the need for 100+ transfer operators to a single operator on Airflow.
Python
Apache Airflow
Data Engineering
ETL
Pest Management System
PyTorch-based model running on phone/cloud to identify harmful pests, alert farmers about infestation and reduce crop loss in cotton farming. Currently used by 11,000+ farmers and contributed to winning Google's $2M AI Impact Challenge.
PyTorch
Computer Vision
Mobile
Agriculture
AI for Good
COVID-19 Mobility Dashboard
Real-time dashboard integrating with COVID-19 Mobility Data Network and Facebook Data For Good, enabling 11 state government authorities in India with actionable insights to optimize emergency protocols during the pandemic.
Python
Data Visualization
Dashboard
Public Health
Interests
I’m broadly interested in Software Engineering, Data Engineering, Computer Vision, and Computer Systems. I’m also a robotics enthusiast and was a Fellow at Teach for India. Always an aspiring polymath!
Achievements & Extracurricular
Winner Design Challenge, ISB Hyderabad (2015) Founded an award-winning startup which built a real-time position reporting system for buses without using GPS. Pitched our idea to Telangana IT minister and received a seed funding offer from TSRTC (Telangana State Road Transport Corporation). [Presentation]
Coordinator, Quality Control And Management - Technozion 15 (Aug 2015 – Jan 2016) Got ISO 9001:2008 certification and UNESCO patronage for Technozion 15, South India’s second largest technical fest. The fest saw a revenue of over 1.5 Crore INR (200k USD) from sponsorships and events. Was responsible for leading, coordination, quality control, risk management, blogging and management for various 150+ events during Technozion 15.
Community Service More than 70 hours of teaching community service in local underrepresented schools as part of Teach for India.
📄 View my complete resume → {#resume}
Recent Updates
2023
- December: Presented “Data pipelines from ETL to ML to LLM: Powered by Apache Airflow” at Microsoft India Development Center Hyderabad with Vikram Koka, SVP of Engineering at Astronomer. [Presentation]
- November: Led development of Ask-Astro, Astronomer’s innovative open-source LLM application integrating Apache Airflow with advanced AI technology
- June: Contributed to Universal Transfer Operator, simplifying data transfer workflows and reducing 100+ operators to a single interface
- Ongoing: Active contributor to Apache Airflow core platform with strong community presence (6K+ LinkedIn followers)
2022
- December: Joined Astronomer as Senior Software Engineer
- August: Presented “Love for writing deferrable operators: Why and how to defer” at Airflow Summit 2022 [Video]
- June: Met Astronomers from around the world in Washington D.C.
- June: Reduced video buffering time by 75% on open-source CVAT annotation tool by implementing Redis cache
- January: Started contributing to astronomer-providers
2021
- March: Google employees join the Wadhwani family on a 6-month Fellowship program. Led engineering effort for annotation studio which is data (artifacts and annotation) management and job orchestration platform to manage, schedule and orchestrate annotation jobs across multiple annotation vendors using Apache Airflow, FastAPI and React
- January: Promoted to Senior Software Engineer at Wadhwani AI
2020
- December: Designed and developed COVID-19 mobility dashboard for 11 state government authorities in India
- May: Published “Pest management in cotton farms: an AI-system case study from the global South” at KDD 2020 [PDF] [Video]
- April: Transitioned to engineering team at Wadhwani AI focusing on Systems and MLOps
- February: Attended a 4-day Google AI Impact Challenge Summit in San Francisco. Had a wonderful time interacting with lots of Googlers and external folks about how they are using AI to build products for societal good. Got lucky to have an unexpected meet with Sundar Pichai!
2019
- September: Invited for Google For India event to showcase AI solution for Early Pest Management in cotton farming and how it can benefit small-holder cotton farmers
- August: Attended 5-day Google AI accelerator in London, UK. Had a wonderful time interacting with lots of Googlers and got feedback on our work with integrated pest management
- May: Part of the team that won a $2 million grant as part of the Google AI Impact Challenge offered to only 20 teams worldwide, becoming the only team from India to do so and featuring in Google I/O and Google for India [Article]
- March: Designed, developed and deployed a pest management system which runs a PyTorch-based model on the phone/cloud to identify the harmful pests, alert farmers about infestation and reduce crop loss in cotton farming. Currently being used by over 11,000 farmers
- February: Joined Wadhwani AI to build AI solutions for social good. Joined as the first software engineer in the team
Contact
For more details, connect with me on LinkedIn • View detailed professional experience →