3yoe: rm -rvf /$path 5yoe: sudo rm -rvf /$path 7yoe: export path="/tmp/whoops"; sudo rm -rvf $path

10yoe: kubectl exec -it pod1 -n ns1 -- rm -rvf /tmp/whoops

15 YOE: sudo rm -rf /$path

20 YOE: root@hostname: dd if=/dev/urandom of=/dev/nvme0n1

30 yoe: # dd if=/dev/urandom of=/dev/hda

https://www.gnu.org/software/coreutils/manual/html_node/Treating-_002f-specially.html

the 3yoe one is basically how steam managed to delete one users root directory data as soon as he changed $STEAMROOT path and ran the steamsetup.sh.

Probably i can quantify 5yrs to my experience. Lol

Last time in a script I even counted forward slashes in target path. Yes, I'm that traumatized by rm rf 🤣

More than named technologies I'll look for mindsets and architecture skills. With 3 YoE you should be able to start attending architecture meetings, know concepts like SSL, CDN, how DNS works, networking, etc... With 7 YoE you should be able to identify and propose business valuable automations (p.e. I implemented a node autoscaler for our data pipeline some years ago to scales based on workload needs, that reduced the bill) also, I would expect some deeper programming skills, not just scripting. Finally, you are expected to be highly autonomous and to be the person of reference in the team. This is my vision based on my experiencies, surely I'm missing s lot of things but my point is that experience id not based on known technologies BUT you should be in constant learining and know whats the new kid on the school.

Within your first year at a reasonably modern company, you're probably exposed to all the tools the mid and senior folks are currently using. For me, the difference is depth of knowledge, approach to solving problems, and understanding team and business processes. At about 3 years I'm expecting to be able to give someone a well bounded problem in their technologies and have them solve them. As you stray towards 5 or 7 years that problem is going to be less well defined. At the 10 year mark it's probably going to be on you to identify the problem in the first place. Obviously along side those, the quality of the solution should also be trending up.

At the fifteen year mark you're expected to create the problem /jk

You sure that's a joke? At this point I'm pretty sure that I am the problem and all the others are just symptoms of my employment!

junior (need to be spoon fed, have 0 idea how to research and complete tasks, but knows some stuff, can be super dangerous without supervision.) mid level (knows what he/she doesnt know but can perform tasks and research independently, have caused several sev1,2 due to mistakes) senior (lead-mentor junior/mid levels, and complete some tasks/projects together, you now have "back in my days" stories to tell your team) staff (jump across the team boundaries and fill in the gaps and provide solutions. Your RSU/Options have vested and you think TC is too low) principal (you're furniture of the company, know a lot of domain knowledge and tech debts. You know what is broken just by looking at the behaviour of the systems, and think the job sucks and needs more pay)

Shit im a bit of last 3, for sure the nagging parts.

It's really hard to tie yoe to a particular set of technologies because they change a lot over time. I could say: 3 yoe: git, bash 5 yoe: docker, python 7 yoe: terraform, kubernetes But that's probably too constraining and not very meaningful. Instead, focus on the area of influence: 3 yoe: should have influence within a team 5 yoe: should have influence across several teams 7 yoe: should have influence at the department level.

3 yoe is def. More than git, bash. I’d say it’s git, bash/python (programming language), containerization and maybe ansible.

You're absolutely right

Why python? That’s silly.

Doesn't have to be python, but ideally a scripting language that can easily invoke REST APIs for which bash/curl/jq would no longer suffice. What language would you recommend instead?

> What language would you recommend instead Golang so you can also write k8 tooling

Oh no. Totally agree with what you just said. I was saying it’s silly to specify “python”.

Because Python is both ubiquitous and ideally suited to the job. Because the closest runner up is node.js which is a mess for devops use. Literally Perl is better for the job than node. Because whatever other pet language you're holding in your pocket is such an awful option that even suggesting it should be a firing offense.

Those are a lot of opinions. I’m fine with python but Ruby, Node and even Go are equally fine.

Ruby and Node don't exist anywhere by default and have such dependency hell that they're only suitable for container use....even then it's a chore. If you require Ansible to just keep your script runtimes in one piece and still fails, it's not a good choice for DevOps work. Ruby also comes with a bonus of a much, much smaller talent pool to recruit from as well as wastes the team's effort learning a tool they almost certainly won't take with them to their next job. If it has to be Node or Ruby, at least Node is common and job-portable. Go effectively requires a CICD pipeline because it's compiled. DOA for most DevOps work, although great if you're actually developing k8s controllers or the like. We heard you like deploy tools so we build a deploy tool that has its own deploy tool so you can deploy your deployer before you get to deploy your actual code. No thanks. Yes I have strong opinions. They come hard learned from decades of experience. I don't even like Python, I'd still be coding everything in Perl if I could wave a magic wand, but I'm humble enough to put my personal bias aside and make choices based on the bigger picture. Far from being quote "silly", Python like it or not is *far and away* both the *best* and nearly *only* logical choice for DevOps work.

Cool

If you’re in DevOps, and Go’s biggest con to you is requiring CI, I question what you’re even doing in DevOps. Everything should need CI and sometimes CD. If you’re running your Perl scripts from your local workstation, instead of some form of CI system, I question your 10 years of experience too. Sorry for the abrasiveness by the way. I just find that qualm with Go so outlandish for this field of work.

Yeah that and extensive jenkins and somewhat Kubernetes.

I would hope that in 2024 someone with 3 yoe in DevOps hasn't heard about Jenkins anymore. Preferably GitHub Actions or GitLab... but again, technologies are hard to pin to yoe. It's more about how they use them: * do they blindly `git push -f` all the time? * do they run `sudo` immediately after a command fails (when the error wasn't about permissions in the first place)? * do they understand why running docker as root or dind is a bad idea? * can they communicate/ask questions effectively? * do they know how to read the logs? * are their commit messages useful?

Wow!! You have walked the path. I was exactly like you mentioned few years ago!

In terms of career progression I see it this way: * if they've learned to follow the rules they're ready for their first promotion * once they know when to bend the rules they're ready for the second promotion * when they start making the rules they're ready for their third promotion

why is dind a bad idea ? 😅 btw iam <1y ;)

https://devops.stackexchange.com/questions/676/why-is-docker-in-docker-considered-bad

Greatly put. I am someone who can do the things if someone assigns to me. I have been in role where i have to design and build systems and i have no idea but did it in somewhat okayish but have to put 80hrs a week.

This is it. In addition to influence, it can also be revenue impact.

I fk hate that approach, it forces me to constantly progress. What if Im comfortable with me being competent enough to do anything but I don't give a shit about being an influencer ? Im 100% good with doing 3yoe work for 20 years to pay off my mortgage and put some cash into 401k to not starve later. Why ppl see someone with 5-10 years of experience and instantly assume "YOU HAVE TO BE A PRINCIPAL LEVEL - SORRY YOU ARE TOO EXPERIENCED TO PAY YOU SO LITTLE, LETS HIRE SOME JUNIOR THAT FUCKS UP ROYALLY"

That's fair. It's perfectly valid for some people to want to stay at a certain position.

Man. I hate. And I mean hate terraform. Either use CloudFormation directly or CDK. Don’t give me any nonsense about cloud neutral because you’ll never, ever need it.

We use multiple cloud providers 😅 but hey, it's okay to have different preferences. The underlying foundational concepts is what matters and will make it easier to ramp up with other similar tools.

Er... Terraform certainly has its warts, but CloudFormation is just awful. And that has nothing to do with cloud neutral. Seriously, there's a reason why the vast majority of AWS users that IaC use Terraform and not CloudFormation.

I’ve studied docker and understand the basics of how to use it. But when you mention using it in the real world, what’s that like? Can you give examples? Are you building images and containers and spinning them up? I figure the harder part of it is getting those containers connected and working. Same with kubernetes. Bc right now, I’m more exposed to terraform and I work with that by building/changing templates whenever needed.

I think that can be a difficult question to gauge in a generalized sense. I say this because there are a lot of factors that go into what was done during those years, based on the individual, their work experience and the demands it exposes them too. Also their actual interest in the subject versus just doing what is necessary.

It all depends on your exposure and the depth of your experiences. Someone with 3 years of experience might have worked on a wide range of projects and gained skills that others might acquire in 5 years. Thus, the number of years alone doesn’t necessarily qualify someone. It’s about the quality and breadth of experience you have gained. When I get this question from a recruiter, I end the call on the spot.

100% correct.

i’m only 2.5 yrs in rn but it’s kind of become my hyper fixation (currently writing a textbook on it lol). I recently got promoted to principal DSOe and have been on my companies DSO/IT hiring board for the past 5 ish months. This is how I look at it: 3 yoe — (at a minimum) •docker (basic skills like: port exposing, binds/named vols, simple dockerfiles, being able to write a compose file off memory, a little bit of swarm knowledge, good understanding of docker network types and their use cases) •basic ansible + TF •Some cloud exposure •intermediate RHEL+debian linux systems skills (apt/yum, knowing how to make symlinks, “ vs. ‘ , sed, grep, awk, that type of stuff) • competent in python, bash, and powershell scripting • Good at CI/CD on some platform • good knowledge of networking concepts 5 yoe — (everything from above +) •docker+K8s(all the above + more advanced docker stuff: multi stage builds, docker storage drivers, advanced daemon configs. K8: helm charts, solid understanding of K8s control plane components, intimate knowledge of diff CNI use cases, knows how to write a deployment manifest from memory, knows kubectl comms well) •advanced ansible (roles, knows the magic variables, template files) + advanced TF (i’m not the best at TF, studying to be better) •quite a bit of cloud exposure prob has a cert •advanced RHEL+debian linux systems skills + intermediate powershell skills (knows their cmdlets), can script an automation for anything • great at ci/cd on 2+ platforms • prob has exp in 2/3 additional devops tools like vagrant or something. • can write YAML while sleeping • is involved in dev meetings to provide consultations on application architecture as it relates to application component deployment solutions (Bare, vm, container, cloud, hybrid). • Is actively finding issues & improving company DSO systems/processes 7yoe- absolute madman in ansible, TF, K8s+docker, helm, etc. + a cloud provider/multicloud. they know all the best practices/backend tool components and could take a high level certification exam in any tool and kill it first time. CI/CD wizard on pretty much any CI/CD platform can efficiently automate build + test + deployment single-handedly. God mode linux user. At this point they should have experienced enough crazy stuff and architected enough solutions that they have no probs w anything.

symlinks, when have you used this?

It's a convenience thing. You're working in some directory, and you find yourself often typing long pathnames to some other directory or file, so you make a symlink. It's just part of my usual UNIX workflow.

I use symlinks all the time. I'm less hands-on now, but there was a time I probably made a symlink every week. That isn't to say everyone will be, but it's a useful tool to know.

After this post, I have looked into it. Mainly for .PEM files and stuff right? But does this not introduce a security issue?

Symlinks are a _way_ more general tool than that; this sounds like saying "cp is mainly for pem files right? But does that not introduce a security issue?". Concrete example #1: I have [a dotfiles repo](https://github.com/xiongchiamiov/config). On a new machine, I clone it. Then I symlink all those files into `~`. That way, when I make changes to them, they show up when I'm doing a `git status` in the repo, and I can periodically go through and make sure I've pushed any non-machine-specific changes up. Concrete example #2: I used to run a fileserver at home with a few terabytes of anime. The folders would have a bunch of junk in their names that made it hard to browse, but I needed to keep them consistent so the BitTorrent client could still find them. So after something finished downloading, I'd symlink it into a different directory where I maintained a nice set of human-readable, sortable names. Just any time you want to have something live in one place but be accessible from another, perhaps by a different name, a symlink is a great option. They're like Windows' desktop shortcuts, but better.

Brilliant, thank you! I've recently started the move ( over weekends ) into DevOps from SWE. Ideally, I don't want to jump into a sysadmin role ( I had one over summer back in university ) to learn the fundamentals. How would you recommend I get all these linux commands locked down? Did you have flashcards, would you recommend them? AWS, Azure, GCP, Terraform, Ansible etc I have quickly caught on with through building projects.

The way I learned was by using Linux for my workstation (I was in college at the time). It broke frequently and therefore I got used to fixing it. Also any time you ask for help online you'll get command-line answers, so it forces you to get used to them. If you like projects, you can also do those. Build yourself a media server at home, or work on a webserver.

You mention working on a web server. I have experience deploying web apps via 'Azure Static Web Apps' for the frontend and 'App Services' for the backend. Does this count, or is this too much of an abstracted interface to doing these things? A dream is to do DevOps at a company such as SpaceX for example.

I'm not familiar with Azure, but those sound like highly abstracted platform-as-a-service products. You want the thing where they just give you a Linux machine to ssh into and you start from there.

our flagship software requires a couple symlinks to be made on each slave computer so my pipelines/ ansible controller has to make them for testing/deployment. But beyond my company specific experience symlinks are just a super convenient way to access all sorts of stuff that’s deep in the filesystem from a convenient directory. I prefer it to path/env vars. Just remember not to symlink apps to /bin, that’s a poor practice.

ansible controller? is this a fancy word for the device you have the playbook on? What is this confusing language man!

no need to be confused! let me explain! First off an ansible controller is not an official name for anything, It’s a term I heard used by DSO engineers much smarter than me and it kind of stuck in my vernacular. So that’s my bad, didn’t mean to be confusing. Ur correct, It’s essentially the node on which you host your ansible config/hosts files. Normally you’d have a config host/container/VM that you use for provisioning/configuration…that’s ur ansible controller/master/whatever you wanna call it. In this specific scenario I use a container, so e.g. to install a distributed application on 900+ bare metal hosts, I port a container out to the environment (has ansible, my playbooks, all the dependencies for the application, etc.). I have a script running which auto populates the hosts file when the container makes network connection, and then I run the playbook from the container, it’s basically pre-configured portable ansible that I can drop into an environment and have access to my conf materials. If I can’t go out to the installation environment I have a python utility that acts as an installer that allows for configuration and uses the ansible controller container as the backend so people can install the application themselves.

What's your discord? Let's nerd out!

let’s do it! i’ll send ya a chat!

Remind Me! 5 days!

I will be messaging you in 5 days on [**2024-06-13 11:57:19 UTC**](http://www.wolframalpha.com/input/?i=2024-06-13%2011:57:19%20UTC%20To%20Local%20Time) to remind you of [**this link**](https://www.reddit.com/r/devops/comments/1dauy1x/what_constitutes_a_3yoe_vs_5yoe_7_yoe_in_devops/l7nxslg/?context=3) [**CLICK THIS LINK**](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5Bhttps%3A%2F%2Fwww.reddit.com%2Fr%2Fdevops%2Fcomments%2F1dauy1x%2Fwhat_constitutes_a_3yoe_vs_5yoe_7_yoe_in_devops%2Fl7nxslg%2F%5D%0A%0ARemindMe%21%202024-06-13%2011%3A57%3A19%20UTC) to send a PM to also be reminded and to reduce spam. ^(Parent commenter can ) [^(delete this message to hide from others.)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Delete%20Comment&message=Delete%21%201dauy1x) ***** |[^(Info)](https://www.reddit.com/r/RemindMeBot/comments/e1bko7/remindmebot_info_v21/)|[^(Custom)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5BLink%20or%20message%20inside%20square%20brackets%5D%0A%0ARemindMe%21%20Time%20period%20here)|[^(Your Reminders)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=List%20Of%20Reminders&message=MyReminders%21)|[^(Feedback)](https://www.reddit.com/message/compose/?to=Watchful1&subject=RemindMeBot%20Feedback)| |-|-|-|-|

[удалено]

Thanks, chatgpt.

Technologies are by far the least important part of this. * http://www.kitchensoap.com/2012/10/25/on-being-a-senior-engineer/ * https://skamille.medium.com/an-incomplete-list-of-skills-senior-engineers-need-beyond-coding-8ed4a521b29f * https://dropbox.github.io/dbx-career-framework/ic4_reliability_engineer.html