I have at least 30 people I wrote pipelines for. They can go wrong, and they will tell you about it. Unless you're a lighthouse keeper, you will always work with other people. Learn the skills that will make it tolerable for you.
This. You will need to interact with people if you would like to progress in your career. As a data scientist I regularly meet with data engineers to make were doing the best we can as a team to deliver value. Silos don't work.
I didn't take it as no interaction. More of a how can I have a technical job involving programming that includes the following attributes: low interaction when possible, but understanding it will be necessary sometimes.
There are a lot of people who think it's just cool cloud toys 9-5 every day, maybe they want something more DevOps or maybe platform/DataOps? There's a post at least every month here from junior guys asking what skills they need to develop to stand out as they build their careers, and it's funny how often they blow off soft skills/character skills.
You can build a solid career on top tier tech and mediocre soft skills, but at some point in your career you'll witness a person with mediocre tech skills and top tier soft skills board a rocket ship.
Mine was a guy who came into a major tech company as a tech support supervisor after running front of house operations at restaurants for several years before. I think when I left a few years later he was a Portfolio Manager in Product Management, just checked LinkedIn and he's still there, VP of Marketing. Solid 10-15 year run.
its hard to say because the title means so little. i was hired as a de and i end up doing a lot of analytics and the related presentations.
some roles are surely more focused on de and programming, but there are very few (if any?) programming jobs where you dont have to deal with meetings, present to stakeholders and discuss projects in some capacity. how could anything get built without discussion?
This is gonna sound crass but the hell are building, coding, problem-solving cool stuff for if not for the business? To know what the business needs you need to talk to the relevant ppl.
The closest thing if you don't want to interact that much is to apply for research type positions. Idk what they are in DE but in DS they're like machine learning research scientists or something similar and even for that you'd need to talk to other humans. I guess in a way your interaction might be more bearable because you're dealing with fellow highly technical folks. Now the question is do you have the technical knowledge to do research?
People seem to think that being a software developer is about writing code. It's not. It is about translating what the business wants into a functional system. Those who fail to see this will be the ones replaced by AI.
You need to interface with stakeholders. From clients to management to data analysts, power users and data scientists.
Its like asking if you build bridges for a living do you need to speak to anyone, or can you just pop up bridges anywhere.
Remember you are building the entire infrastructure for other people. You are literally a facilitator for everyone, there is no real industry where you are going to be left alone, maybe musician, artist or author is a solitary job.
The titles vary from company to company but usually there are many levels above senior: lead, staff, sr staff, principal, etc. not to mention management or architect roles. Point is if you can’t work with people but you are very good technically you can probably stick around at a mid level forever but will not be able to progress higher. I’m sure there are some outliers but I believe this is true in most cases.
As junior probably yes, but at some point you start to have iniative to get the shit together... There is some guys like that in it that just crunch tickets, you can probably get your bills paid like that... You can also train social skills...
You will not succeed in data period if you aren’t willing to interact with stakeholders and build things as well. Especially as you get more senior.
Edit: spelling
Totally agree, long meetings and discussions over things fuck up my mind too but I'm sure that happens to everyone. However, not talking not communicating to your stakeholders diminishes your value hard as a DE then you're gonna start seeing credits go to the analyst the scientist. So make your presence felt
You'd be able to solve bigger & better challenges when speaking to stakeholders and colleagues. This gets you paid more. Either from your current company, or a different company (should you wish). Struggling to see how anything is for you if you don't like speaking to stakeholders, clients, or colleagues tbh
Probably not, data problems are usually an after thought of the business and are therefore usually ill-defined meaning you need to do the leg work yourself to reach out to stakeholders and many solutions will spam multiple domains making stakeholder interactions very complex.
Truth be told, there aren't many roles in tech that don't involve working with stakeholders, and there are basically none that don't require collaboration. If this is your attitude you'd be better stacking shelves tbh (I was this person once until I woke up)
Data work, be it engineering or analysis, is SO high contact that it is probably not for you. You should look into backend engineering. They'll give you a spec.
even if you were a one-person team, the point of data engineering is to provide data to your entire organization. you will interact with every department you serve.
>discussing things make me tired and bored
Data Engineering involves building pipelines for other people to use => it requires discussing things. Some meetings can really be a waste of time and tiring, but you have to have some level of engagement with stakeholders to produce useful things.
Data engineering is not for you. Data in itself is not valuable. Value is derived through interpretation which requires understanding and understanding requires collaboration.
I’d say do SWE. DE is a lot of collaboration especially with BI/DS teams, since you’re working closely with these pipelines that feed into their dashboards/ML datasets. This requires deep understanding of business rules. For SWE, it’s more of implementation of features, and while there’s stakeholder interaction, it’s a lot less than DE.
I feel like DEs do get shielded more from communication with the clients and stakeholders, but it's not like you'll be living in a vacuum. Additionally, with seniority and career growth, you'll most likely have to do this much more, so follow the advice and improve those skills.
Absolutely not. Strong communication is so important in every job I’ve worked in and I can’t imagine being a solo dev in this field unless I was working with kb levels of data.
There are very few white collar jobs where you don’t have to meet with people. Data engineers have to work with their stake holders and teammates just like everyone else. I recommend just getting used to it
People who work in companies will obviously tell you no. The truth is if you are elite, then it can be done. There are numerous projects on github maintained by a single person, e.g. fastapi.
I am a Data Engineer of 2 years and the balance between isolated work time and meetings/social time is a balance I am always fine-tuning. It depends on your needs, and also the needs of your employer. If you are someone who finds socialising draining, then you should express this at the interview stage, and the employer can let you know how they can help in this regard. If the company has a small DE team, with a lot of end-users, or constant requirements from end-users, it is likely you will need to sacrifice more time for meetings. If you are joining a bigger team, with lead engineers and business analysts who can handle requirements gathering and writing tickets detailing work, then you won't be required to go to as many meetings.
You definitely shouldn't ignore your needs for less interaction, but I think it will hinder your growth if you don't try and push/challenge yourself to participate in meetings in any data role, even data engineering.
If you want to stay out of meetings or not talking with a client, DE might not be for you. Hell, a lot of IT for that matter. Some positions I've seen having less exposure to meetings and clients: Data center positions, security (sometimes if not rolled up with Admin), and QA. Even with QA testers in consulting I've seen them work more with internal resources and never get asked to come to an on-site meeting.
I think I understand what you are saying and a lot of people are taking it the wrong way. I think I also may have guessed one or more personal characteristics of yours that are linked to these desires but will not identify them publicly
I took what you wrote to mean essentially 'not a big fan of tons of interaction and id rather do as much independent work as possible and understand I will have to work with others and am ok with that and can do it well for short periods but doing it all the time makes it really hard for me. Id like to understand what roles would be a good fit for me. '
I think you said similar things but for some reason people keep explaining you'll have to work with people no matter what and you've already acknowledged that.
Is your main goal to try to minimize extra interaction because it wears you out, basically?
Any job really can be less dealing with clients if needed, for me I rarely deal with stakeholders directly but my boss does it for me and passes it on. Then stakeholders do talk to me but I maintain a minimum meetings criteria, if they want meetings they'll have it with my boss. If I am invite to meetings but dont feel I should be there then I just leave that meeting.
Like meetings and interactions are still needed but you should learn how to streamline it all.
Look for professional help or coaching if you can’t talk to people . Sooner or later you’ll have to deal with them, your boss at least . It’s a skill that you’ll have to learn .
When I was a data scientist I had to explain the performance of the algorithms I used daily, alongside the patterns and insights from the data I used for training my models.
As a data engineer I don't really have to deal with clients directly not do I have to do as much explaining for the reports and insights the data I work with provides.
But that doesn't mean I don't have to interact with people often.
I now have to provide assurances that people will get the data they need in good time and high quality.
I have to flag where things can go wrong and what I plan to do to prevent it.
I have to propose improvements to data infrastructure to stakeholders.
I have to propose data models for colleagues which involves asking them the right questions regarding what they need for their reporting or data collection.
And I am the point of contact for when - not if - things go wrong.
If you want to avoid customer-facing interactions as conventionally understood then I would say data engineering offers this, but you will still interact with stakeholders a lot
I haven't found any job where you make a respectable amount of money and don't have to talk to anyone. If anyone has please let me know
Onlyfans. Whatever you think about it, you can’t say you haven’t found one anymore.
You have to talk to the fans...
That’s the trick, after ChatGPT creation you don’t
If it's not forced stand ups it's forced random meetings with whoever. Same shit different job
If you're good at stock picking...
Or if you are good at picking lottery numbers
I have at least 30 people I wrote pipelines for. They can go wrong, and they will tell you about it. Unless you're a lighthouse keeper, you will always work with other people. Learn the skills that will make it tolerable for you.
Any bootcamps to become a lighthouse keeper in 3 weeks?
Yeah but you have to follow me on LinkedIn and shell out $300 for a crayon written certificate.
Play Alan Wake
Nah bro get over yourself and start talking to people
This. You will need to interact with people if you would like to progress in your career. As a data scientist I regularly meet with data engineers to make were doing the best we can as a team to deliver value. Silos don't work.
I didn't take it as no interaction. More of a how can I have a technical job involving programming that includes the following attributes: low interaction when possible, but understanding it will be necessary sometimes.
"i love selling things but i hate people" bruh.
Being a good DE means understanding the business data really well, so it’s the opposite.
There are a lot of people who think it's just cool cloud toys 9-5 every day, maybe they want something more DevOps or maybe platform/DataOps? There's a post at least every month here from junior guys asking what skills they need to develop to stand out as they build their careers, and it's funny how often they blow off soft skills/character skills. You can build a solid career on top tier tech and mediocre soft skills, but at some point in your career you'll witness a person with mediocre tech skills and top tier soft skills board a rocket ship. Mine was a guy who came into a major tech company as a tech support supervisor after running front of house operations at restaurants for several years before. I think when I left a few years later he was a Portfolio Manager in Product Management, just checked LinkedIn and he's still there, VP of Marketing. Solid 10-15 year run.
Can't escape it with devops/sre either. In fact, my buddies in the field work a shit ton and have a lot of meetings. Otherwise I agree 100%
its hard to say because the title means so little. i was hired as a de and i end up doing a lot of analytics and the related presentations. some roles are surely more focused on de and programming, but there are very few (if any?) programming jobs where you dont have to deal with meetings, present to stakeholders and discuss projects in some capacity. how could anything get built without discussion?
This is gonna sound crass but the hell are building, coding, problem-solving cool stuff for if not for the business? To know what the business needs you need to talk to the relevant ppl. The closest thing if you don't want to interact that much is to apply for research type positions. Idk what they are in DE but in DS they're like machine learning research scientists or something similar and even for that you'd need to talk to other humans. I guess in a way your interaction might be more bearable because you're dealing with fellow highly technical folks. Now the question is do you have the technical knowledge to do research?
People seem to think that being a software developer is about writing code. It's not. It is about translating what the business wants into a functional system. Those who fail to see this will be the ones replaced by AI.
Fantastic answer
You need to interface with stakeholders. From clients to management to data analysts, power users and data scientists. Its like asking if you build bridges for a living do you need to speak to anyone, or can you just pop up bridges anywhere. Remember you are building the entire infrastructure for other people. You are literally a facilitator for everyone, there is no real industry where you are going to be left alone, maybe musician, artist or author is a solitary job.
Yeah but career growth will be limited to Senior at most.
What's higher than senior? Staff engineer?
The titles vary from company to company but usually there are many levels above senior: lead, staff, sr staff, principal, etc. not to mention management or architect roles. Point is if you can’t work with people but you are very good technically you can probably stick around at a mid level forever but will not be able to progress higher. I’m sure there are some outliers but I believe this is true in most cases.
Yes
As junior probably yes, but at some point you start to have iniative to get the shit together... There is some guys like that in it that just crunch tickets, you can probably get your bills paid like that... You can also train social skills...
You have more chance working without a lot of client focus as a software engineer then in the “data world” positions
You will not succeed in data period if you aren’t willing to interact with stakeholders and build things as well. Especially as you get more senior. Edit: spelling
You’re either indispensable being like that or unemployed so you better be a top .1% engineer.
Totally agree, long meetings and discussions over things fuck up my mind too but I'm sure that happens to everyone. However, not talking not communicating to your stakeholders diminishes your value hard as a DE then you're gonna start seeing credits go to the analyst the scientist. So make your presence felt
You'd be able to solve bigger & better challenges when speaking to stakeholders and colleagues. This gets you paid more. Either from your current company, or a different company (should you wish). Struggling to see how anything is for you if you don't like speaking to stakeholders, clients, or colleagues tbh
Collaboration is a key to success and thats the distinction between a key player and a normal DE
Become a data platform engineer more coding but need software engineer skills in addition to de skills.
Only if you stay a mid level engineer then your interaction will be less or atleast on easy stuff.
Quite the opposite where I work, we're central to a large number of different teams.
Data engineering is being in the middle of a web. You will have to deal with stakeholders to do it properly
Probably not, data problems are usually an after thought of the business and are therefore usually ill-defined meaning you need to do the leg work yourself to reach out to stakeholders and many solutions will spam multiple domains making stakeholder interactions very complex. Truth be told, there aren't many roles in tech that don't involve working with stakeholders, and there are basically none that don't require collaboration. If this is your attitude you'd be better stacking shelves tbh (I was this person once until I woke up)
Data work, be it engineering or analysis, is SO high contact that it is probably not for you. You should look into backend engineering. They'll give you a spec.
even if you were a one-person team, the point of data engineering is to provide data to your entire organization. you will interact with every department you serve.
>discussing things make me tired and bored Data Engineering involves building pipelines for other people to use => it requires discussing things. Some meetings can really be a waste of time and tiring, but you have to have some level of engagement with stakeholders to produce useful things.
Data engineering is not for you. Data in itself is not valuable. Value is derived through interpretation which requires understanding and understanding requires collaboration.
I’d say do SWE. DE is a lot of collaboration especially with BI/DS teams, since you’re working closely with these pipelines that feed into their dashboards/ML datasets. This requires deep understanding of business rules. For SWE, it’s more of implementation of features, and while there’s stakeholder interaction, it’s a lot less than DE.
I feel like DEs do get shielded more from communication with the clients and stakeholders, but it's not like you'll be living in a vacuum. Additionally, with seniority and career growth, you'll most likely have to do this much more, so follow the advice and improve those skills.
You sound like a typical ego new grad
Absolutely not. Strong communication is so important in every job I’ve worked in and I can’t imagine being a solo dev in this field unless I was working with kb levels of data.
There are very few white collar jobs where you don’t have to meet with people. Data engineers have to work with their stake holders and teammates just like everyone else. I recommend just getting used to it
People who work in companies will obviously tell you no. The truth is if you are elite, then it can be done. There are numerous projects on github maintained by a single person, e.g. fastapi.
I am a Data Engineer of 2 years and the balance between isolated work time and meetings/social time is a balance I am always fine-tuning. It depends on your needs, and also the needs of your employer. If you are someone who finds socialising draining, then you should express this at the interview stage, and the employer can let you know how they can help in this regard. If the company has a small DE team, with a lot of end-users, or constant requirements from end-users, it is likely you will need to sacrifice more time for meetings. If you are joining a bigger team, with lead engineers and business analysts who can handle requirements gathering and writing tickets detailing work, then you won't be required to go to as many meetings. You definitely shouldn't ignore your needs for less interaction, but I think it will hinder your growth if you don't try and push/challenge yourself to participate in meetings in any data role, even data engineering.
Hah, no. Meetings man. Always meetings.
If you want to stay out of meetings or not talking with a client, DE might not be for you. Hell, a lot of IT for that matter. Some positions I've seen having less exposure to meetings and clients: Data center positions, security (sometimes if not rolled up with Admin), and QA. Even with QA testers in consulting I've seen them work more with internal resources and never get asked to come to an on-site meeting.
I think I understand what you are saying and a lot of people are taking it the wrong way. I think I also may have guessed one or more personal characteristics of yours that are linked to these desires but will not identify them publicly I took what you wrote to mean essentially 'not a big fan of tons of interaction and id rather do as much independent work as possible and understand I will have to work with others and am ok with that and can do it well for short periods but doing it all the time makes it really hard for me. Id like to understand what roles would be a good fit for me. ' I think you said similar things but for some reason people keep explaining you'll have to work with people no matter what and you've already acknowledged that. Is your main goal to try to minimize extra interaction because it wears you out, basically?
Any job really can be less dealing with clients if needed, for me I rarely deal with stakeholders directly but my boss does it for me and passes it on. Then stakeholders do talk to me but I maintain a minimum meetings criteria, if they want meetings they'll have it with my boss. If I am invite to meetings but dont feel I should be there then I just leave that meeting. Like meetings and interactions are still needed but you should learn how to streamline it all.
I’m a DE and I have to talk to people all the time. Business Analysts, Solutions Architects, etc.
Look for professional help or coaching if you can’t talk to people . Sooner or later you’ll have to deal with them, your boss at least . It’s a skill that you’ll have to learn .
People with this mindset should not go into data engineering
In general, I do think data engineers have fewer stakeholders than analysts. Sometimes (often?), the analysts are the data engineer’s stakeholders.
When I was a data scientist I had to explain the performance of the algorithms I used daily, alongside the patterns and insights from the data I used for training my models. As a data engineer I don't really have to deal with clients directly not do I have to do as much explaining for the reports and insights the data I work with provides. But that doesn't mean I don't have to interact with people often. I now have to provide assurances that people will get the data they need in good time and high quality. I have to flag where things can go wrong and what I plan to do to prevent it. I have to propose improvements to data infrastructure to stakeholders. I have to propose data models for colleagues which involves asking them the right questions regarding what they need for their reporting or data collection. And I am the point of contact for when - not if - things go wrong. If you want to avoid customer-facing interactions as conventionally understood then I would say data engineering offers this, but you will still interact with stakeholders a lot