Data Science Sea: Swim or Drown?

Why am I writing this?

First of all hello to all the readers, it is good to be back on Medium after so long. If you don’t remember me or have not read any of my articles, as an introduction I am a Senior Data Scientist working for a well-known FMCG giant, and below are some of my previous articles. https://medium.com/@commondatascientist/thought-it-would-be-fun-i-was-wrong-data-science-interview-2d8b43cbd94c https://medium.com/mlearning-ai/practical-experience-is-what-matters-in-data-science-d05bc3018717

The stage where I am in the journey of Data Science always makes me wonder what next. I assume this question comes to most of the minds in this field, so I thought of writing my thoughts on the platform to connect with you all, gain some learnings from your experience, and share mine. As a disclaimer, the things you read from now are completely my thoughts and you may find that it does not resonate with you, but again this is what makes this platform a truly diverse one isn't it? While writing, I will also post the questions that are in my mind and I am not able to get the answers as of now.

Data Science Sea: https://medium.com/@guljabeen222/data-science-and-the-many-streams-db9bdddcff16

Thanks to one of fellow writer for the above image ,it shows a almost perfect representation of how vast the Data Science is, you feel this might not be complete as there are lot of new tech and streams getting added to data science almost every year. In the age of Chat GPT there will never be a perfect representation of the above streams

Who Are You?

Commonly, your next set of actions is defined by your present stage. Let us go through different stages of any career for any professional and discuss what can be the Options, Skills to acquire, and Decisions.

Disclaimer :I will be discussing these stages for an engineer because that is what I have been

I won’t be discussing the Fresher stage where the person is just coming out of college, talking about the population mostly the decision in this stage is governed by the initial CTC and the company you are getting into.

Experience: 2 to 3 years

Questions :

1. Should I swim into the sea of Data Science?- Being in this field, I receive calls from my known ones for advice on how they can switch to the data science stream, the thing is they have already made up their mind to switch because this field has been the hottest stream in recent times and it makes sense to get into if you can. Please ask yourself, do you love solving the problems that can impact decisions in the company vision? Do you love numbers and mathematics? Do you have the urge to learn at least one programming language and excel? If the answer is no to any of the questions then I would suggest reconsidering your decision
2. Job Switch?- Let’s say you are already working in the field of Data Science as a Data Analyst, Consultant, etc. If the purpose is to switch jobs then you will find several guides on the net that can tell you how to prepare and you should be able to do it within a month at this stage of your career. The suggestion is to switch your first job within 2–3 years, it truly doesn’t let your knowledge become stagnant and also gives your career the push financially.
3. Data Science or Data Engineering?- Most common question to have at this stage of the career, the answer depends on what you like as a person.

Data Engineer: If you love to code more and build pipelines, design databases, and get involved more on the architecture side of things, love the cloud side of things more, this might be the path for you

Data Scientist: Closer to the business problems, love to see problems in maths, want to build the conclusions of the problems and present to the business, code a bit, this is the path for you

A term that is very prevalent nowadays like ML Engineer, we will come to them in the next stage of our career.

Skills to Acquire : Till this point most of the skills are common which ever path you chose

SQL(Must), Python(Numpy, Matplotlib, Pandas), Sckit LearnBasic Statistics, Case Study Solving, Puzzles, Bit of Cloud Computing

Is this too deep?

Experience: 4–8 years

Now, this is the critical phase of a career where a lot of questions come your way and it mostly paves the next 5 years for you. There will be a lot of skills which you should acquire in this phase.

The next set of actions for you is also defined by what you are doing in your existing job role.

Questions

1. Artificial Intelligence, Deep Learning, NLP, Generative AI, etc. where to swim?- These terms will always intimidate you in the age we are living in, and it sure should do. Things are happening in the world that may be beyond our imagination right now in AI, so this phase is the right time for you to learn new stuff and maybe go into a specific area of Data Science that is more appealing to you. I am not saying that there are no companies that can provide you with multiple skills at work parallel but there are very few of them that allow you to do that. Once you go into this niche stream the coding you have done up till now will also change and you will be introduced to a newer set of algorithms and frameworks. There are a lot of startups that are working in these niche areas and will provide you ample opportunities if you want to flourish in these fields.

To remind you, whatever skills you have learned till now will be useful in any of the streams you decide to choose.

All the above-mentioned tech streams are subjects in themselves and there are different skills you require to solve the problems specific to each of them. Skills and Frameworks: Linear Algebra, Applied Mathematics,NLP core techniques, Computer Vision, Frameworks like Keras, Tensorflow, Pytorch(Must), Neural networks understanding. Solve more and more problems on the platforms like Kaggle, this will help you to switch your job to these areas.

The problem with the above streams is that they demand your complete dedication, you have to learn a lot of things and experiment with a lot of things before you can help the business in the end and build a product

2. ML Engineer?- You might have heard that 80% of your projects as a data scientist don’t reach the production stage, I would say this is half a truth, the project might not be reaching the production stage as the answers that you were trying to find might not require a complete production-ready model. It might just require you to run an ML algorithm on the historical data and find certain answers through that. The other reason for not reaching the production stage is that you don’t want to spend time deploying the model and building a pipeline over it as it is just a monthly model and you can always go and run the notebook manually whenever the new data comes in.

ML Engineers :)

Now here comes the role of ML engineers, they are mostly the extended versions of the data scientists with a lot of overlaps but organizations now prefer a person who can build the model end to end starting from the data collection to the production-ready model, they are sometimes also called as full stack data scientists

Skills to Acquire: Comfortable with Cloud, MLOps Framework(Must), DevOps and CI/CD, Scala, Pyspark(Other Big data languages like HIVE, PIG), Version Control

There are certain skills which are relevant any work which you do at this stage in your career, for example: Writing a production level code(Must), familiar with SQL(Advanced joins), Optimization of the code etc.

In terms of earning potential ML engineers do earn more than a Data Scientist on an average depending on the organization as well.

Experience: 8–12 years

This is where I am in my journey and I have a lot of questions that are unanswered till now, this phase will govern mostly the remaining part of your career, as they say, once you grow to a certain level of experience it is tough to switch.

Questions

1. Manager or Independent Contributor(IC): A very intriguing question in every job profile is, do you want to lead a team? There are many instances I have seen in my circle where a person becomes a manager and finds it tough to gel with the team as it requires a different set of skills for a manager to become a good leader. I won’t say one is better than the other as it depends on how you as a person are. If you go a path of IC you will be working with different teams but you will have a defined set of responsibilities, Principal architects is an example of this job profile

If you good at working with people and have the ability to utilize the strong set of a particular individual to the project cause , mentor individuals, balance client and team expectations then you should be leading a team, and if you are comfortable working alone with your work not depending much on the others, you should go on the path of IC.

Skills to Acquire: People Management, Scalable Archietecture, Desigining , Project Management, Client First

2. Which Industry to swim in?: Now is the time to choose an industry you want to spend the next few years in. There are a lot of industries and the decision depends on your previous years' experience and the learnings you got from it.

FMCG, Fintech, Supply Chain, Education, Logistics, Healthcare, and Cybersecurity are some examples where great work is going on in Data Science, it is not that you cannot switch from one industry to another after certain years of experience, it is just that you would want to be in an industry which you like in the prime of your career, and it will always be tough for an organization to take you at a higher position from a completely different industry

This is a decision which I have still not taken in my career but the only thing you need to remember in this phase is to learn everyday about the business you are into and try to read and read more about the new things happening in the world of Data Science

Ending Notes

I have not written anything about the option which is open to everyone at any stage in their career, it is to start on your own or the well-known word Startup, let us leave this topic for another day.

You might come to these questions a bit earlier than what I have defined in your career, but the thing is to take the leap of faith and not be confused in whatever you do.

Please share your thoughts, questions, dilemma in comments and try to help each other as a community.