Protect SSH login of your linux server with fail2ban (Centos 6)

If you have a public Linux server on internet, you can see in your log (/var/log/secure) there are tons of people from everywhere trying to login in to your server to get control of your machine. Massive brutal attacks is a common practice. Even if they are not a big issue they are anyway quite annoying. If the attack rate is really high, it can start to be a performance issue.
Let’s try to improve the security of you server:

1. Keep your server update, in particular keep updated network software, accessible services (sshd, smtpd, snmp, etc) and related security libraries  (OpenSSL etc)
2. remove password based ssh login: There is a dedicated post about Secure SSH Linux Login
3. remove root login: look at the same post about Secure SSH Linux Login
4. install fail2ban to block brutal force attacks

Install Fail2ban in Centos/RHEL

1. What is Fail2ban and how it works
   fail2ban is an application monitoring your logs looking per specific anomalies. For example too many failed login from the same IP, meaning somebody not authorized is trying to login into your server. Once a pattern is discovered (example: too many failed login), and action is taken. The typical action is block the IP attacking your server using iptables for a defined period (ip are often dynamic, so no sense to block it forever) but any type of action can be taken by Fail2ban (send email or sms, launch a script, restarting an application, etc). A common use is the ssh login protection, blocking IP with many failed login, but Fail2Ban can manage many different application log (smtp, http, etc)
2. Enable iptables
   normally in my servers iptables (the Linux firewall) is disabled, because I have external devices to filter and control traffic. If the security control is inside the same server, in case of any break into a server they can change the security rules and access to something else, if the security is managed outside changing the rules is much more complex. Anyway to make Fail2ban effective, we need the support of iptables. Iptables is a critical component, so pay attention to modify it, you can lock yourself out. Let’go:
   Check if iptable is running:
   [~]# service iptables status
   Table: filter
   Chain INPUT (policy ACCEPT)
   num  target     prot opt source               destination
   …….

   In case is not running, start Iptables

   [~]# service iptables start

   and check if everything is still running: in case you are lock out, just reboot the server (you need ILO/remote manager or a physical access to server).
   In case it is runing without any problem, make sure it will start on next boot

   [~]# chkconfig iptables on
3. Install Fail2Ban
   first you need to be sure in your linux box Epel repository is registered:
   [~]# rpm -Uvh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm

   Epel is a repository for application supported directly by Fedora project and indirectly by RedHat. We can consider it reliable (when you install an application from this repo you are quite secure it is the original one, not hacked or compromised). install fail2ban with yum

   [~]# yum install fail2ban

   in case of any error during installation or running the application, maybe it can be due some missing dependencies, so please install ipset with

   [~]# yum install ipset
4. Configure Fail2ban
   Your custom configuration will be located in a local configuation file. So copy the original to create your own copy
   [~]# cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local

   Now setup some details

   [~]# vi /etc/fail2ban/jail.local

   in particular how long you want an ip to be banned, the list of your known ip so fail2ban will never ban them, default frequency of errors (in case you can redefine case by case)

   [DEFAULT]
   # “ignoreip” can be an IP address, a CIDR mask or a DNS host. Fail2ban will not
   # ban a host which matches an address in this list. Several addresses can be
   # defined using space separator.
   ignoreip = 127.0.0.1
   # “bantime” is the number of seconds that a host is banned.
   bantime = 21600
   # A host is banned if it has generated “maxretry” during the last “findtime”
   # seconds.
   findtime = 1800
   # “maxretry” is the number of failures before a host get banned.
   maxretry = 3

   Some detail:
   ignoreip: contain the list of your known ip, separated with a space, you can use single ip or range like 192.168.1.0/24.
   bantime: the time an IP is banned. We suggest to take 21600 (6 hours), maybe you can chenge it to 43200 (12 hours)
   findtime/maxretry: if the same IP makes maxretry times in findtime seconds a failed login, it is banned. We suggest 3 times in 1800 seconds
   Remember you can repeat this settings inside each control to customize these values.
   Now enable sshd check setting “enable” in your config file

   [ssh-ddos]

   enabled  = true
   filter   = sshd
   action   = iptables[name=SSH, port=ssh, protocol=tcp]
   sendmail-whois[name=SSH, [email protected], [email protected]]
   logpath  = /var/log/secure
   maxretry = 3

   The name of the jail (ssh-ddos) can be any. If you want to receive an email when an IP is locked, remember to set your email address as “dest” and put a sender address (always use an existing address as sender). If you do not want receive any email, just comment out this line.
   Then simply start the service

   [~]# service fail2ban restart

   You will find in your messages log files all log related to fail2ban, IPs banned and IPs unbanned

   [~]# grep fail2ban /var/log/messages
5. TEST IT!!!
   Many times I realized fail2ban was not blocking what It was supposed to block due to wrong configuration, wrong log format, wrong log position and so on. So using a server with a different ip from yours, try to login via ssh with wrong user/pwd and check if your ip is blocked. It take about a minute to block your IP but you can see what is happeneing inside the message log.
   If you do not see any particular problem, make your fail2ban persistent in case of reboot
   [~]# chkconfig fail2ban on
6. Maintenance
   Remember in case you restart iptables, you need to restart fail2ban immediately after. Syncerely I suggest to change the script to start/restart Iptables (/etc/init.d/iptables) to run fail2ban immediatel after iptables start.

Side effects

I used this software so many time in the past, in so many different environments, so I can tell you some strange things can happen due to particular environments.

1. Massive locking: an example: we used this tool to protect email serves, but if someone changes his password without updating all his clients, the clients looks like an attacker continuously trying to login with the previous wrong login settings. If it is done from the office, the ip of the office is banned so all the people in the  office can not access anymore. So remember to set the whitelist in “ignoreIp”, only one user can close all the office if the office ip is not whitelisted. You can have the same effect with other shared ip you do not control. If someone, inside a large company, with all people surfing internet with the same IP, tries to access in a wrong way to a service in your server protected by fail2ban and the IP is banned, this means nobody inside this company will be able to access to that specific service on your server.
2. Private vs public ip: before setup fail2ban pay attention of your network configuration. If your server has a private ip inside a network receiving public traffic through a device (firewall, proxy, whatever) take a look to logs before starting fail2ban. If the device in front of your server is a layer 7 device, probably you are logging the private ip of the device, not the public ip of the clients, so in case of any block, you are blocking the private ip of your network device and so all the traffic coming to your server, not only the bad ip. It can happens typically using fail2ban to block bad http traffic. Take a look to the logs to be sure you are logging real public ip, not the one of you proxy or firewall