What is The Big Data Conference all about?
The Big Data Conference is a technology meetup that is devoted to the field of Data Engineering. It places a strong emphasis on delving deep into Data Engineering problem statements and discussing them in detail.
The conference provides a great opportunity to network with other passionate engineers who share the same love for their craft. Attendees can learn from insightful talks and, most importantly, have fun.
Planned Talks
Apache Hudi 1.0 preview: A database experience on the data lake
By: Sagar Sumit (Apache Hudi Committer & Software Engineer at Onehouse)
Join the OneHouse team, creators of Apache Hudi, for an insightful talk on the open-source data management framework. Discover how Hudi is a versatile data lake platform that enables automated, near-real-time data ingestion and incremental processing. It is seamlessly integrated with powerful frameworks such as Apache Spark, Flink, and Kafka Connect. Learn about its key features, real-time capabilities, and how it empowers organizations to build robust, scalable, and reliable data pipelines. We will explore the current state of the project, Hudi 1.0 Preview, and the road ahead planned by the Hudi community.
Facilitating Real-Time Streaming with Kafka and Apache Spark for massive dataset & scale
By: Ashutosh Kumar (Staff Software Engineer PayPal)
Empower your real-time data processing with our Kafka and Spark-driven streaming solution. Utilizing Kafka for robust data ingestion and Spark for seamless analytics, our application enables continuous data flow, from ingestion to insightful results. Avoid pitfalls by scaling effectively, ensuring fault tolerance, monitoring performance, and optimizing processing logic.
Streamlining Data Complexity: Unveiling the Power of One Big Table in Modern Database Design
By: Ankit Bansal(Senior Engineer Amazon & NamasteSQL founder)
'One Big Table' - a revolutionary approach in database design. Discover how consolidating data into a unified structure simplifies querying, enhances performance, and transforms the landscape of modern data management. Uncover the power of streamlined data architecture in this engaging session.
Streaming and Real-Time - Uncharted territory for Lakehouse Engines
By: Faiz Kothari & Sudarshan Lakshminarasimhan (Senior Performance and Research Engineer at E6Data)
Streaming and near-real-time ingestion capabilities on Hudi, Delta, and Iceberg unlock exciting and high-impact use cases on data lakehouses. However, while these solve for ingestion performance and efficiency, they present uniquely difficult challenges for distributed and MPP engines that query such data. Uncover the design thinking of developers at e6data outlines an unconventional approach to engine architecture, metadata management, and distributed execution in overcoming such challenges.
Panel Discussion
Navigating Data Engineering Challenges: Insights from Startup Experts
Moderator: Ankur Ranjan (Founder of The Big Data Show)
Panelists: Gopi Krishna (VP of Data Engineering at Angel One) & Aparnesh Gaurav(
Engineering leadership | Data platforms & softwares | Swiggy )
Join us for an insightful exploration into the world of Data Engineering within startup ecosystems. Delve into the pivotal role of Data Engineering in driving startup success, uncovering challenges, fostering cross-collaboration, acquiring talent, navigating tech stacks, ensuring data ethics and compliance, and discovering avenues for continuous learning and professional growth.
Journey into Data Lakehouse: Unveiling the Third Generation of Data Design
Moderator: Ayush Bijawat (Senior Data Engineer at Walmart)
Panelists: PadmaPriya Uppala (Senior Data Engineer at Visa) & Nilesh Mahajan(Head of Platform at Onehouse)
Embark on an illuminating journey as our panel of experts unveils the paradigm-shifting concept of Data Lakehouse, representing the cutting-edge third generation of data design. Join us to explore its evolution, strategic significance, technical challenges, and transformative impact on analytics and business decisions. Dive into discussions on adoption strategies, governance, security, and the immense potential this innovative model holds for reshaping data infrastructure and unlocking new possibilities in the digital landscape.
Organizing Team
Ayush Bijawat Senior Data Engineer
Ayush Bijawat Senior Data Engineer
Ayush Bijawat Senior Data Engineer
Sai Vineel Senior Data Engineer
Ayush Bijawat Senior Data Engineer
Ayush Bijawat Senior Data Engineer
Shakti Naik Senior Data Engineer
Ayush Bijawat Senior Data Engineer
Shakti Naik Senior Data Engineer
Kuldeep Pal Data Engineer III
Ayush Bijawat Senior Data Engineer
Shakti Naik Senior Data Engineer
Shweta Jaiswal Senior Data Engineer
Shweta Jaiswal Senior Data Engineer
Shweta Jaiswal Senior Data Engineer
Ankur Ranjan Data Engineer III
Shweta Jaiswal Senior Data Engineer
Shweta Jaiswal Senior Data Engineer
Supported By
Venue Partner
