Published in100 Days Data·PinnedData Engineering Beyond HypeLet’s try to dig into Data Engineering core concepts beyond the hype. Let’s face it, Data Engineers are no longer boring plumbers of data moving it from one place to another. Sure, Data Scientist was the sexiest job post in the last decade, and prompt engineers, are massively hyped right…Data Engineering2 min readData Engineering2 min read
Published inData Engineer Things·PinnedHow to Create First Data Engineering Project? An Incremental Project RoadmapBuild Data Engineering projects in this incremental approach for guaranteed success — On Apr 16, 2022, I wrote a detailed Roadmap of how to learn Data Engineering as a beginner. The roadmap was a great hit. However, it had a significant gap. People often fail to remain consistent while learning so many different technologies. They fail to piece it together. The roadmap…Data Engineering15 min readData Engineering15 min read
Published inCodeX·PinnedHow to Become a Data Engineer : Complete RoadmapA complete roadmap on how you can learn Data Engineering in 2022 — In October 2012, HBR predicted that Data Science will be the sexiest job in the 21st century. For the first 10 years of the century it did seem to be exactly matching. However companies soon realized that without proper data infrastructure in place and without quality data, data science projects…Data Engineering8 min readData Engineering8 min read
Published inCodeX·PinnedIs Data-Engineering a Great career move in 2022?Data Engineering is the new Sexiest Job of the century — In October 2012, HBR predicted that Data Science will be the sexiest job in the 21st century. For the first 10 years of the century it did seem to be exactly matching. The highly quoted report was perfect to predict a huge surge in demand for Data Scientists as resources…Data Engineering5 min readData Engineering5 min read
Published inCodeX·Apr 1What’s The Real Value of Data For Business?Data Engineering Beyond Hype #2 — As businesses become more data-driven, the importance of data has increased exponentially. But what is the real value of data for business? In this article, we will explore the impact of data on various industries through real-life case studies. We will also discuss what data professionals can learn from these…Data Engineering7 min readData Engineering7 min read
Published inCodeX·Sep 4, 202260+ Years of history in Data ProcessingData Engineering Beyond Hype #1 — This is a light read on the history of Data Analysis and processing over the years. However, at times light reads give so much perspective into the Why and How it was done a certain way, and How far we have come from the moment we started. Let’s try to…Data Processing4 min readData Processing4 min read
Published inCodeX·Aug 30, 2022Data Pipeline Architecture: a variety of ways you can build your Data PipelineData Engineering: Beyond Hype #6 — Data Pipelines are the main building block of Data Lifecycle Management. Data Engineers spend 80% of their time working on Data Pipeline, design development and resolving issues. Since this is so important for any Data Engineering workload, let’s try to dive deeper in al the different architectures to build a…Data Engineering9 min readData Engineering9 min read
Published inCodeX·Aug 18, 20227 Best Practices for Data IngestionData Engineering: Beyond Hype #5 — “Data Engineering is the new Sexiest job in 2022” It has surpassed Data Science in demand and career opportunities. If you have not already seen the astronomical growth in demand for Data Engineering, chances are you were living in a cave for the last 2 years. What exactly is the…Data Engineering6 min readData Engineering6 min read
Jul 19, 2022What is Data Engineering?What is Data Engineering? It is 50% of time trying to answer…. Why is this data wrong? Why the Data is not matching with the source? 40% of time contemplating…. What am I doing in this Data Sea? Why I am even here? 10% of time writing Kick ass code…1 min read1 min read
Jul 8, 2022Migration in Data Engineering: A MythMigration in Data Engineering means moving data from one place to another. A common myth in Data Engineering is that Data Engineers only move data from one place to another. And that’s the only migration they are ever involved in. This common myth has created a false belief that: Data…2 min read2 min read