Dancing your way through the pathless data career
Exploring the how and why behind my non-linear journey in data and its resonance with many in the field.
Data roles definition has ben hard to understand since the beginning. There’s no traditional path, people are coming from different background and each companies have their own nuanced definition. Add on top of the that the rapid evolution of our industry and these roles changing and the confusion even get bigger.
In this post, we're going to look at how we got to this point. I'll try my best to offer you a guide to get through this confusing situation and create your own successful career in data without following a set path.
My pathless career in data
Below is my career path over the last decade. As you can see, I have held multiple different job roles. While it may not be as unconventional as some paths I have seen, where people transition from industries such as HR, filming, or music, it serves as a good example to show that you should not be afraid of taking such a path. In this blog, I will share a couple of personal experiences and tips that will hopefully help you in your data journey.
The flaws in the classic career scheme
A classic career path seems simple in some industries.
Want to be a doctor? You go to med school.
Want to be a lawyer? You head to law school.
In the world of data, things are a bit more complicated. Through my mentoring experience, I've noticed that young graduates or those shifting careers often express a desire to work in data, AI, or ML. However, they usually don't have a clear idea of what they want to do. They might have some guesses about their interests, but mostly, they just want to break into the industry.
That career path is not straightforward at all.
Another scenario is when you're already employed in the data field and aim to shift to a different role, for example, transitioning from Data Analyst (DA) to Data Engineer (DE). This transition, too, is far from straightforward.
If you check out my career journey, you'll see that my first data job was an internship. I left a steady job and took a 20% pay cut to grab a special chance to learn a lot and set myself up for later success. It's alright to start from scratch. Once you're in the industry, moving up becomes a lot easier.
Let’s draw our plan based on what the market needs
When you look at data engineering job posts on LinkedIn, you can see this:
Even for jobs with the same title, they ask for totally different basic skills.
So what’s going on here ?
Understanding everyone’s data maturity
Data maturity can be defined in many ways. I like to think it as your ability to use data to make business decisions that boost profit or productivity. At the very end, it’s not just about the tech tool stack or your code, but it’s for business.
One issue is that big tech companies like Netflix and Airbnb are openly sharing their tech know-how, which is generally a good thing. However, this makes other companies think that achieving similar tech levels is easier than it actually is. They often don't realize how wide the tech gap really is.
About a decade ago, in 2012, there was a rush to embrace Machine Learning, leading to a surge in hiring Data Scientists. However, many companies soon realized they couldn't effectively utilize their data.
The image above, from a 2015 Google paper, illustrates the average time a Data Scientist spends on an ML model.
Interestingly, despite being a global trend, many companies still find themselves in this scenario today, each experiencing their own "aha" moment when they realize its relevance to them.
So, should we simply label ourselves as data fellows and dive into data tasks?
That approach doesn't really assist anyone in charting their career path. To some extent, role titles are important.
Reflecting on my career journey, I made the leap into the tech scene about four years ago. Prior to that, I was employed in more traditional companies. While securing a job in such places might be easier and you might not get to use all the latest tech tools, they offer a solid foundation. They provide a great starting point for building skills and breaking into the data field.
Dissecting roles
Everyone has probably come across a version of the well-known data roles Venn diagram.
Different versions of this Venn diagram exist. The issue is that they imply skills are strictly linked to certain jobs. For example, they make it seem like a Data Scientist never does ETL (Extract, Transform, Load) work. But in reality, Data Scientists often do engage in ETL tasks.
The reality is, skills often overlap a lot between different roles, but how much they overlap can change from one company to another. This diagram doesn't show how deeply you need to understand each skill for each job. This can be overwhelming, given the huge range of skills involved.
A more accurate way to represent this might be through a web chart (or radar chart), as shown below.
The further away you go from the center, the higher the value is. This graph is not to be taken for gold as a one source of truth but as a good baseline.
This graph enables us to comprehend the variations in job offers for identical roles. So you could potentially have
And even this :
Because sometimes, even the company is not sure about which role they want to hire and they put too many requirements. They don’t realize they are not looking at one person role, but a whole data department.
Back in 2015, when I began working in the data realm, the term 'data engineer' wasn't really popular. I initially rode the wave of excitement into a Data Scientist role, but soon realized it wasn't quite the right fit for me. Don’t hesitate to try different things, as shifting between roles is often smoother than making your initial entry into the field.
What You Need to Learn to Break Into Data
"The art of knowing is knowing what to ignore." – Rumi
To thrive in the data field, it's essential to focus on a mix of both hard and soft skills. Here's a non-exhaustive list that balances technical abilities with personal competencies, offering long-term value and market relevance:
Data Analysis: Interpreting and understanding data.
Programming (SQL/Python/etc): Critical for data manipulation.
Data Visualization: How to tell a story and present data insights visually.
Software engineering Basics: Understanding the software development lifecycle, from building code to deploying it into production.
Data Warehousing Concepts: Knowing about data storage and management.
Critical Thinking: For solving problems and making decisions.
Communication: Data roles are in the center of many stakeholders with different background, communication is challenging.
Adaptability: Staying flexible in a rapidly evolving tech landscape.
When building your skill set:
Focus on Long-Term Value: Opt for skills that will stay in demand. Be cautious about becoming too specialized in certain tools or technologies that might not stand the test of time.
Market Relevance: Align your skills with the current needs of the job market. Regularly check job listings to stay informed about desired skills.
These skills can be applied across various data roles. Customize your learning based on these essentials, keeping in mind your career objectives and the ever-changing job market.
When I transitioned to a data engineer role, I immediately recognized that my SQL skills were transferable. However, it was also clear that I lacked foundational knowledge in software engineering. I was eager to delve into this area, and now, with a better understanding of how software functions beyond just the data aspect, I can see the immense value of these skills. They're not just useful; they're transferable across various domains.
Another piece of enlightening advice in the same line comes from Steph Ango's recent blog, "Don't Specialize, Hybridize." As illustrated in the image below, he demonstrates that there are multiple pathways to gaining expertise, going beyond the traditional two-path approach.
Beyond just getting a job
We've mostly talked about landing a job, but it's also about finding joy in your work. How do you stand out? How do you find something you truly enjoy?
Enter Ikigai, a Japanese concept that's all about finding your purpose.
We've focused a lot on the 'what you can be paid for' part, but it's important to discover what you love and what you're good at.
So, how can you make your own moves?
Explore industries you're passionate about.
Think about your soft skills. What are your strengths? What do others appreciate in your personality?
Find activities that feel like fun to you but work to others. Blending these with your professional life can really add some excitement.
Reflecting on my own career, I noticed I spent a lot of time blogging, making YouTube videos, and teaching. These were not just hobbies; they helped me stand out and led me to a new role in developer relations (devrel).
A great way to experiment with this is through side projects, and they don’t have to be tech-related. Anything that lets you play with your interests and figure out what you're good at can be beneficial.
The key challenge lies in understanding how these pursuits can enrich your professional life.
Creating your own map
Navigating a career in data can feel like a labyrinth. Unlike traditional paths, it demands a nuanced understanding of both technical and business aspects.
The market needs are constantly evolving, and so are the definitions and expectations of various roles within the industry. Moreover, the pace of change varies across organizations.
Craft your unique, pathless career and avoid being too fixated on specific job roles:
Focus on transferable skills.
Align these skills with the demands of the market.
Reflect on what activities you genuinely enjoy and mix them with your work.
Flexibility, continuous learning, and self-awareness are your keys to unlocking extraordinary opportunities.
Good luck! 🍀
Great article. I have a very similar career journey as yours, switched from data science to data engineering and didn’t look back!