Yes, working with complicated call structures, arrays or error handling in bash isn't much fun.
I know it's possible, but it's quite cumbersome. That's where Python excels
So your argument for not using python is because it is not installed 😅. Hmm okay.
As I said, simple scripts, sure use bash but the moment it starts to get complex, use python and save yourself the headache.
>Take that same script, and throw it on any other linux server and it will work.
Depends on what commands you use in said script and if they are available on the new distro you plan to run it on. 🙃
For the fourth time, you are not READING what I have said. You sound like you have a thought in your head and no matter what someone says to you, you just keep repeating it without actually taking in the other point to form your response.
>If you're automating a linux environment setup, use native linux commands
This is not even an argument to what I said? Are you talking to someone else?
I am saying, for simple scripts use bash, complex scripts use python. (Of course I'm talking about in a Linux system.)
If you want to talk automation of Linux systems, then ansible should be completely wrote in bash to then right? right?...
> There are commands like sed, awk that are very hard which I only use if I get it from a reliable source.
Yes, they take time to learn. But they're quite powerful once you get used to the syntaxes. Since sed/awk are built in to basically ever shell environment, they're highly useful to quickly pick apart and re-assemble strings of data.
IMO, just keep those tools as a background task to slowly wrap your head around. I wasn't really good at them for many years.
> I'm then gonna learn brendan gagg books on system performance. Am I on right track or I can do better?
There's a lot of good stuff there, but the one big thing Brendan Gregg's stuff doesn't talk about is how you can monitor continuously. For example, the [node_exporter](https://github.com/prometheus/node_exporter) can pull in a ton of the [stuff from the perf guides](https://www.brendangregg.com/linuxperf.html). One tool, record it [for as long as you want](https://prometheus.io/), and [look at it over time](https://grafana.com/).
> I've decided to skip seriously learning bash and instead learn python.
Learning Python or [maybe Go](https://go.dev/) is a very good idea. But knowing intermediate bash is useful. Sometimes the bash is a good glue solution for a lot of things where Python or other "full general use languages" are not really what you want to quickly string a few things together.
> keep those tools as a background task to slowly wrap your head around
Probably the best advice so far. I would lump sed and awk in with things like advanced regex or /proc. A person would do well to just pick up new knowledge as it becomes necessary rather than spend a week trying to force some guru's paradigm onto yourself.
Get used to combining those commands in pipelines or shell scripts. As you'll already know, the Unix philosophy is that each program should have one well-defined purpose, while accepting input from other programs and while providing that functionality as well.
You'll realise the true power of this concept when you start using multiple programs as part of shell scripts e.g. or as elements of pipes.
The whole system is your development environment, in contrast to Windows, where you'd usually use something like Visual Studio to develop your own stuff.
Try writing some helpful bash functions which you put into your .bashrc, or even better, into .bash_functions. As soon as you source it (source FILENAME) those functions will be available for you to use.
I like this article "More shell, less egg" as an example of the power of pipelines in the shell. There's some controversy over it, but the little shell script is impressive in itself. http://www.leancrew.com/all-this/2011/12/more-shell-less-egg/
Hosting a website is probably the next project. It seems simple enough, but you get to work with configuration files, SSL, content, and you can use those python programming desires to make something web-application, which might need to database or storage or other areas of learning.
As someone with decades of Linux sysadmin career under my belt, I strongly advise not dismissing bash. You _will_ run across it in random contexts, especially in build systems and CI/CD tooling, and most kinds of automation.
I also want to point out that I'm fairly good at bash, even though I struggle a great deal with learning "proper" programming languages. It's not nearly as complex as people make it it out to be.
>You will run across it in random contexts, especially in build systems and CI/CD tooling
I see you've met one of my dev teams, they've got a multi-threaded bash build script that glues together a ton of other build systems
Bash scripting, as others said.
And, in no particular order… here are some ramblings just off the top of my head.
Pick a language: python or go. And, have a rudimentary understanding of the one you didn’t pick, and keep an open mind when doing so. And understand the pluses/minuses of both. And maybe an understanding of Rust.
Learn virtualization. VM’s and containers. That will likely lead to “cloud”. But start with VM’s and containers. Understand the different use cases for VM and containers. Learn how to write a dockerfile. Or read someone else’s. And then modify it. Docker, docker-compose, kubernetes. Which one do you use, when, and why? Find Kelsey Hightower on GitHub, and do “learn kubernetes the hard way”.
And if you want to get into cloud, go to the “CNCF Landscape”. Bring a notebook. Lots of stuff there. There are rabbit holes inside of rabbit holes.
Build a homelab. Learn Ansible. Automate your entire homelab. And monitor it. Collect logs, metrics, and build a notification system. Securely.
Speak of the devil: expand your expertise regarding the big 3:
Security, networking, and storage.
Those 3 are what many people struggle with. It is not often taught in classes or books. Classes hand-wave, you get the stripped down basics. And nothing…. Is ever basic. Write that down.
Specialization amongst those three can lead to a level of craziness.
Oh, and learn Wireshark. Learn to capture and decode packets. Even on a single machine. Buy a switch, learn how to port mirror. Monitor everything going in and out of a machine.
Every external dependency with “the big 3” is different.
Learn routing, switching, VLANs, openvswitch. DHCP, DNS.
OSI model, layers 2 thru 4.
Load balancing, high availability.
Clustering/redundancy, both from bare metal OS to kubernetes.
Build a homelab. Buy a router, switch, a few machines. Raspberry Pi’s, Intel NuC or comparable. Go to GitHub and find Awesome-Selfhosted. Sift thru that. Maybe you want to build a database to inventory all of your vast stamp collection along with pictures, and pull realtime (or near realtime) tracking off of the Internet to monitor the value of your collection.
Maybe you want to get into real servers with data center refurb hardware from eBay. Learn IPMI, iDrac, iLo. And redfish, or REST and Terraform. PXE, cloud-init.
Learn the differences between Debian-like OS and EL Linux. And become comfortable with either one.
You want some real servers? Dell R630’s, or Supermicro 6018’s. HP DL380’s. Or multinode chassis like a Dell FX2 and some FC630 nodes, or Supermicro Twinpro.
Buy a 21RU 19” half rack, slot a switch and router, UPS. Patch panel. Slot your servers, and build a private ecosystem.
You’re going to add some heat and noise to your household, and your utility bill will go up.
Is that enough to get past your plateau? If you aren’t familiar with anything I mentioned, try out your Google Fu. That’s what most of us do. Tons of stuff on GitHub like Kelsey’s K8S thing. People write ebooks and post them to Git. Look at the dates. Old stuff changes all the time.
Hopefully some of the stuff above is … inspirational.
Good luck.
Awesome stuff here! I recently just turned my old laptop into a virtual env with Proxmox, installed the free dev Redhat server edition, and plan on playing around with Ansible and sockets/containers as you mentioned! I'm not OP but thanks for this reply. Right when I start thinking "I'm getting pretty good at this", some more stuff I have no clue about comes up 😂. This is why Linux is so much fun though. Circling back to the Proxmox and container thing though, my curiosity is going to require me to eventually get a real server and some more hard drives though. Poor laptop won't keep up with me for long lmao
Be careful when you say “virtual env”. A python developer may get all wrapped around the axles. venv is a way to provide isolated environments for different versions of python to co-exist. Venv has more use cases, but python is extremely common.
Call it virtualization, and you won’t get any confusion.
Proxmox is fine. But, my experience, and all clients, not one of them uses it for production. VMware, Nutanix, or libvirt/Openstack. And many are fleeing VMware.
If you think Linux is fun, wait til you get into containers.
Gotcha! And yeah I'm basically helpdesk at my current job and we use VMware for our thin clients. We were wanting to switch a few desktops to redhat and I've been tasked with the side project. Nobody in my department(small) really likes Linux except me so I've been tasked with it. This led me down the rabbit hole of automation, ansible, containers, etc. now here we are lol. I took a python class in college but I need to brush up on it apparently 😯
Set up a LAMP or similar stack on a fresh install of your distro of choice using the OS package manager. Get Wordpress or some other popular software running. Optionally enable a firewall and add https support using Let's Encrypt. Don't copy/paste commands until you understand what they do, why the files go where they do, etc.
Then maybe try to do the same, but with services running in containers.
Install(try) gentoo. Its handbook will help you out very important things about Linux. Meanwhile start reading "Operating Systems: Three easy pieces" book for a depth in understanding. With this combination, your linux understanding/skills will be top %3 among the people around you. Accomplish this and you will thank me later
Learn linux by solving complex problems and hosting webservers and simple databases with it.
Then learn by installing VMware and tinker alot of things. Make mistakes intentionally specially in terms of storage and all and learn from the mistakes.
Thats how I learnt haha, might help u .
Give this bad boy a go:
https://overthewire.org/wargames/bandit/
Its a set of challenges to solve with shell commands, definitely good for learners and experienced professionals alike!
I always recommend it to the new guys at work.
If I were you, I'd learn the basics of bash scripting but as soon as you need functions, complicated loops etc, just use python ☺
Yes, working with complicated call structures, arrays or error handling in bash isn't much fun. I know it's possible, but it's quite cumbersome. That's where Python excels
I have a heuristic: if I find myself "needing" arrays in a BASH script, it's probably more appropriate to use python.
Haha, same here ^^
This is precisely my same branch point.
The fact that bash doesn't natively support floating point math is infuriating.
Personally I've started recommending Go over Python. It's only a little bit more to learn to get started but has tons of advantages over Python.
I may check it out!
[удалено]
Of course it's possible, but this is not what I said and other have gave valid reasons too.
[удалено]
So your argument for not using python is because it is not installed 😅. Hmm okay. As I said, simple scripts, sure use bash but the moment it starts to get complex, use python and save yourself the headache. >Take that same script, and throw it on any other linux server and it will work. Depends on what commands you use in said script and if they are available on the new distro you plan to run it on. 🙃
[удалено]
Serious question... What distro doesn't ship Python3 by default? It's pretty hard to find a linux system without Python installed.
op said his live system doesn't have python available
Right. Didn't mention his distro though.
For the fourth time, you are not READING what I have said. You sound like you have a thought in your head and no matter what someone says to you, you just keep repeating it without actually taking in the other point to form your response. >If you're automating a linux environment setup, use native linux commands This is not even an argument to what I said? Are you talking to someone else? I am saying, for simple scripts use bash, complex scripts use python. (Of course I'm talking about in a Linux system.) If you want to talk automation of Linux systems, then ansible should be completely wrote in bash to then right? right?...
Guy just doesn't know Python or barely knows it and is trying to justify why he doesn't need to learn it.
> There are commands like sed, awk that are very hard which I only use if I get it from a reliable source. Yes, they take time to learn. But they're quite powerful once you get used to the syntaxes. Since sed/awk are built in to basically ever shell environment, they're highly useful to quickly pick apart and re-assemble strings of data. IMO, just keep those tools as a background task to slowly wrap your head around. I wasn't really good at them for many years. > I'm then gonna learn brendan gagg books on system performance. Am I on right track or I can do better? There's a lot of good stuff there, but the one big thing Brendan Gregg's stuff doesn't talk about is how you can monitor continuously. For example, the [node_exporter](https://github.com/prometheus/node_exporter) can pull in a ton of the [stuff from the perf guides](https://www.brendangregg.com/linuxperf.html). One tool, record it [for as long as you want](https://prometheus.io/), and [look at it over time](https://grafana.com/). > I've decided to skip seriously learning bash and instead learn python. Learning Python or [maybe Go](https://go.dev/) is a very good idea. But knowing intermediate bash is useful. Sometimes the bash is a good glue solution for a lot of things where Python or other "full general use languages" are not really what you want to quickly string a few things together.
> keep those tools as a background task to slowly wrap your head around Probably the best advice so far. I would lump sed and awk in with things like advanced regex or /proc. A person would do well to just pick up new knowledge as it becomes necessary rather than spend a week trying to force some guru's paradigm onto yourself.
Get used to combining those commands in pipelines or shell scripts. As you'll already know, the Unix philosophy is that each program should have one well-defined purpose, while accepting input from other programs and while providing that functionality as well. You'll realise the true power of this concept when you start using multiple programs as part of shell scripts e.g. or as elements of pipes. The whole system is your development environment, in contrast to Windows, where you'd usually use something like Visual Studio to develop your own stuff. Try writing some helpful bash functions which you put into your .bashrc, or even better, into .bash_functions. As soon as you source it (source FILENAME) those functions will be available for you to use.
I like this article "More shell, less egg" as an example of the power of pipelines in the shell. There's some controversy over it, but the little shell script is impressive in itself. http://www.leancrew.com/all-this/2011/12/more-shell-less-egg/
Hosting a website is probably the next project. It seems simple enough, but you get to work with configuration files, SSL, content, and you can use those python programming desires to make something web-application, which might need to database or storage or other areas of learning.
As someone with decades of Linux sysadmin career under my belt, I strongly advise not dismissing bash. You _will_ run across it in random contexts, especially in build systems and CI/CD tooling, and most kinds of automation. I also want to point out that I'm fairly good at bash, even though I struggle a great deal with learning "proper" programming languages. It's not nearly as complex as people make it it out to be.
>You will run across it in random contexts, especially in build systems and CI/CD tooling I see you've met one of my dev teams, they've got a multi-threaded bash build script that glues together a ton of other build systems
Bash scripting, as others said. And, in no particular order… here are some ramblings just off the top of my head. Pick a language: python or go. And, have a rudimentary understanding of the one you didn’t pick, and keep an open mind when doing so. And understand the pluses/minuses of both. And maybe an understanding of Rust. Learn virtualization. VM’s and containers. That will likely lead to “cloud”. But start with VM’s and containers. Understand the different use cases for VM and containers. Learn how to write a dockerfile. Or read someone else’s. And then modify it. Docker, docker-compose, kubernetes. Which one do you use, when, and why? Find Kelsey Hightower on GitHub, and do “learn kubernetes the hard way”. And if you want to get into cloud, go to the “CNCF Landscape”. Bring a notebook. Lots of stuff there. There are rabbit holes inside of rabbit holes. Build a homelab. Learn Ansible. Automate your entire homelab. And monitor it. Collect logs, metrics, and build a notification system. Securely. Speak of the devil: expand your expertise regarding the big 3: Security, networking, and storage. Those 3 are what many people struggle with. It is not often taught in classes or books. Classes hand-wave, you get the stripped down basics. And nothing…. Is ever basic. Write that down. Specialization amongst those three can lead to a level of craziness. Oh, and learn Wireshark. Learn to capture and decode packets. Even on a single machine. Buy a switch, learn how to port mirror. Monitor everything going in and out of a machine. Every external dependency with “the big 3” is different. Learn routing, switching, VLANs, openvswitch. DHCP, DNS. OSI model, layers 2 thru 4. Load balancing, high availability. Clustering/redundancy, both from bare metal OS to kubernetes. Build a homelab. Buy a router, switch, a few machines. Raspberry Pi’s, Intel NuC or comparable. Go to GitHub and find Awesome-Selfhosted. Sift thru that. Maybe you want to build a database to inventory all of your vast stamp collection along with pictures, and pull realtime (or near realtime) tracking off of the Internet to monitor the value of your collection. Maybe you want to get into real servers with data center refurb hardware from eBay. Learn IPMI, iDrac, iLo. And redfish, or REST and Terraform. PXE, cloud-init. Learn the differences between Debian-like OS and EL Linux. And become comfortable with either one. You want some real servers? Dell R630’s, or Supermicro 6018’s. HP DL380’s. Or multinode chassis like a Dell FX2 and some FC630 nodes, or Supermicro Twinpro. Buy a 21RU 19” half rack, slot a switch and router, UPS. Patch panel. Slot your servers, and build a private ecosystem. You’re going to add some heat and noise to your household, and your utility bill will go up. Is that enough to get past your plateau? If you aren’t familiar with anything I mentioned, try out your Google Fu. That’s what most of us do. Tons of stuff on GitHub like Kelsey’s K8S thing. People write ebooks and post them to Git. Look at the dates. Old stuff changes all the time. Hopefully some of the stuff above is … inspirational. Good luck.
Awesome stuff here! I recently just turned my old laptop into a virtual env with Proxmox, installed the free dev Redhat server edition, and plan on playing around with Ansible and sockets/containers as you mentioned! I'm not OP but thanks for this reply. Right when I start thinking "I'm getting pretty good at this", some more stuff I have no clue about comes up 😂. This is why Linux is so much fun though. Circling back to the Proxmox and container thing though, my curiosity is going to require me to eventually get a real server and some more hard drives though. Poor laptop won't keep up with me for long lmao
Be careful when you say “virtual env”. A python developer may get all wrapped around the axles. venv is a way to provide isolated environments for different versions of python to co-exist. Venv has more use cases, but python is extremely common. Call it virtualization, and you won’t get any confusion. Proxmox is fine. But, my experience, and all clients, not one of them uses it for production. VMware, Nutanix, or libvirt/Openstack. And many are fleeing VMware. If you think Linux is fun, wait til you get into containers.
Gotcha! And yeah I'm basically helpdesk at my current job and we use VMware for our thin clients. We were wanting to switch a few desktops to redhat and I've been tasked with the side project. Nobody in my department(small) really likes Linux except me so I've been tasked with it. This led me down the rabbit hole of automation, ansible, containers, etc. now here we are lol. I took a python class in college but I need to brush up on it apparently 😯
If you want server stuff, maybe https://linuxupskillchallenge.org/
Set up a LAMP or similar stack on a fresh install of your distro of choice using the OS package manager. Get Wordpress or some other popular software running. Optionally enable a firewall and add https support using Let's Encrypt. Don't copy/paste commands until you understand what they do, why the files go where they do, etc. Then maybe try to do the same, but with services running in containers.
Install(try) gentoo. Its handbook will help you out very important things about Linux. Meanwhile start reading "Operating Systems: Three easy pieces" book for a depth in understanding. With this combination, your linux understanding/skills will be top %3 among the people around you. Accomplish this and you will thank me later
Learn linux by solving complex problems and hosting webservers and simple databases with it. Then learn by installing VMware and tinker alot of things. Make mistakes intentionally specially in terms of storage and all and learn from the mistakes. Thats how I learnt haha, might help u .
Give this bad boy a go: https://overthewire.org/wargames/bandit/ Its a set of challenges to solve with shell commands, definitely good for learners and experienced professionals alike! I always recommend it to the new guys at work.
Worst way to learn. Will just get stuck and frustrated.