Howdy folks. Phil your friendly neighborhood security guy here with a new series on securing your servers and systems. This series will be focusing mainly on Linux systems, Docker Containers, and Vulnerability management - Oh My!
Your systems are under attack
Cisco data estimates that distributed denial-of-service (DDoS) attacks will grow to 15.4 million by 2023, more than double the 7.9 million in 2018.
Over 4/5th of data breaches in 2020 were financially motivated, according to Verizon’s 2020 Data Breach Investigations Report (DBIR).
According to research by IBM, it takes 280+ days to detect and mitigate the average cyber attack, with an average attack costing $3.86 million
You are important
You might be thinking your site isn't that important to be attacked or there is nothing of value there. This is the wrong type of thinking. Personal server attacks are on the rise, and not for what you think. Attackers will use anything they can to get the advantage, and your IP, server space, and domain names are juicy targets.
You aren't the only one looking for a great spot on the internet to host your applications. Attackers do it as well. Since 2017, there has been a huge uptick in Crypto-mining campaigns. Normally we see these attacks on WordPress sites running outdated / unsupported plugins such as this one or this one. I have even seen F5 devices get popped and running Crypto mining software. One such investigation lead me to find 2 wallet IDs with 3 miners running on it. They were pulling in over $160 a week from a few Digital Ocean boxes. Don't worry, I contacted Digital Ocean and they are shutdown now.
Clean IP reputation
IPs are one of the largest and most specific indicators of compromise around. ARIN numbers don't change often, and IP's typically don't change Geo Location or IP owner. As such, your servers IP is a great resource to attackers because it's most likely NOT on threat list. There are tons of open source threat lists out there, and many routers leverage them to actively block attackers from rogue IPs. Your server most likely isn't on the emerging threats list or Infragard. As we can see below, even Linode and Digital Ocean servers usage is on the rise.
Attackers don't like setting up infrastructure. As someone who has setup C2 infrastructure and written malware that interacts with it, I can tell you, it's a pain. As such, attackers won't just send out links directly to their systems. They often times will use URL re-directors hosted on popped servers. This allows them to link to your server, and redirect to their server. If your server happens to get shutdown or flagged, they don't care as it's your site, not theirs.
But - I'm not a security expert
Your right, you most likely are not a security expert, and that's the reason for writing this series. With security requiring such a wide range of skill sets and knowledge, it can be a daunting task to get into. More over, most self hosting people are only just now getting to understand how something works, let alone how to secure it. However, armed with this series of posts, you will be able to look for and identify suspicious behavior. Most of the time security is just knowing where to look and what's "weird".
Rogue Process Detection
You didn't think I'd make you read all that without some type of reward right? Let's start with something simple. What's running on your server? Let's look for rogue processes and strange things. Some easy commands to look are below.
Searching for weird
Many people will tell you to just run a command like
ps aux and sift through the lines and look at everything, but unless you are a Linux expert, it's a little daunting. Instead, let's run a nice little utility I like called pstree.
sudo apt install -y psmisc
Once that's complete, lets run
This list is much more manageable. This is a typical server running a few typical processes. you can see sshd (Open SSH Server), containerd (Docker), Systemd (init.d replacement and standard on most newer Linux machines) and even Auditd (we will talk about this later). This gives you not only a list of processes, but also what spawned them in an easy to read interface. Go ahead, run this command on your server and see if you see anything strange. Google the strange thing and see what it might be, or leave a comment.