Hi, I'm Ankit

I am a Senior Software Engineer at Astronomer, where I work on design and engineering of Apache Airflow (an open source workflow management platform). Astronomer is the orchestration platform for the modern data stack. Built on Kubernetes, Astronomer makes it easy to run, monitor, and scale Apache Airflow clusters in our cloud or yours.

I am a strong advocate of open-source, and believe in Learning by Doing. I have contributed to Apache Airflow, Ask-Astro, Universal Transfer Operator, Astro-Python-SDK, Astronomer-Providers and OpenCV CVAT repositories.

Prior to this, I was leading engineering team engineering teams across multiple projects in the healthcare and agriculture domains at Wadhwani Institute for Artificial Intelligence which is an independent, non-profit research institute and global hub, developing AI solutions for social good. I am more of a generalist with strong backend skills. I enjoy working with all layers of the stack and have a strong belief in automation. I have implemented a complete backend, different microservices accross the projects, data and annotation pipeline for a pest management project for which Wadhwani AI won Google’s AI impact challenge and got a 2 million grant from Google. Just a fun fact, I got a chance to a demo application in front of Sundar Pichai as part of Google AI Impact challenge.

I have previously worked with startup focused on AR/VR camera applications and SDKs, drone data analysis and was also founding engineer at a startup in the home improvement space where I worked on building web based solutions that leveraged classical ML models.

Prior to that, I obtained my B.Tech at NIT Warangal where I founded an award (Design Challenge by Indian School of Business Hyderabad) winning startup which built a real time position reporting system for buses without using GPS.

I am skilled in Python, C++, SQL, NoSQL, Apache Airflow, Django, FastAPI, Data Engineering, Analytics, and Reporting, leading engineering teams, SDLC, and hiring. I am broadly interested in Software Engineering, Vision, Data Engineering and Computer Systems.

I am also a robotics enthusiast and was a Fellow at Teach for India. Always an aspiring polymath!

My CV can be found here.

Updates:

December 2023 : Talk on Data pipelines from ETL to ML to LLM: Powered by Apache Airflow (presentation) at Microsoft India Development Center Hyderabad with Vikram Koka, SVP of Engineering at Astronomer.

November 2023 : Led the development of Ask-Astro, Astronomer’s innovative open-source LLM application, integrating Apache Airflow with advanced LLM technology to create a versatile chatbot and knowledge interface, leveraging Retrieval Augmented Generation and Weaviate vector store for enhanced AI-driven query response capabilities in Airflow and Astronomer domains.

June 2023 : Worked on Universal Transfer Operator that simplifies how users transfer data from a source to a destination using Apache Airflow. It offers a consistent agnostic interface, improving the users’ experience so they do not need to use explicitly specific providers or operators. This operator reduces the need for 100+ transfer operators to the single operators on Airflow.

Aug 2022 : Talk on Love for writing deferrable operators Why and how to defer (talk recording)

June 2022 : Meet Astronomers from around the world in Washington D.C.

Jan 2022 : Started the contributions astronomer-providers

Dec 2022 : Thrilled to join Astronomer as a Senior Software Engineer.

June 2022 : Reduced the video buffering time by 75% on open-source CVAT annotation tool by adding Redis cache to cache the frames during data ingestion.

March 2021 : Google employees join the Wadhwani family on a 6-month Fellowship program. Lead engineering effort for annotation studio which is data (artefacts and annotation) management and job orchestration platform to manage, schedule and orchestrate annotation job across multiple annotation vendors using Apache Airflow, FastAPI and React.

Jan 2021 : Promoted to Senior Software Engineer at Wadhwani AI

Dec 2020 Designed and developed a covid-19 mobility dashboard integrating with COVID-19 Mobility Data Network and Facebook Data For Good which enabled 11 state government authorities in India with actionable insight to optimize emergency protocols in line with epidemic spread and identifying high movement areas.

May 2020 : Our work Pest management in cotton farms: an AI-system case study from the global South got accepted at KDD 2020. [PDF] [Video]

April 2020 : Transitioned from research to the engineering team at Wadhwani AI to focus on Systems and MLOps.

February 2020 : Attented a 4-day Google AI Impact Challenge Summit at San Francisco. Had a wonderful time interacting with lots of Googlers and external folks about how they are using AI to build products for societal good. Got lucky to have an unexpected meet with Sundar Pichai!

September 2019 : Invited for Google For India event to showcase our AI solution for Early Pest Management in cotton farming and how it can benefit small-holder cotton farmers.

August 2019 : Attented a 5-day Google AI accelerator at London, UK. Had a wonderful time interacting with lots of Googlers and got feedback on our work with integrated pest management.

May 2019 : Part of the team that won a $2 million grant as part of the Google AI Impact Challenge offered to only 20 teams worldwide, becoming the only team from India to do so and featuring in Google I/O and Google for India.

March 2019 : Designed, developed and deployed a pest management system which runs a pytorch based model on the phone/cloud to identify the harmful pests, alert farmers about infestation and reduce crop loss in cotton farming. Currently being used by over 11,000 farmers.

Feb 2019 : Joined Wadhwani AI to build AI solutions for social good. Joined as a first software engineer in the team