Securing a collocated Raspberry Pi

There’s an updated version available of this guide here

This is a small guide i wrote for setting up a new Raspberry Pi server hosted at PCExtreme I know there are plenty of guides on how to secure your server, but i wanted this as a recipe for myself.

I don’t normally use “cookbooks”, but since setting up a remote server isn’t as trivial as setting up a local server, i felt it was worth creating a list of things to do, afterall if things go wrong you can’t just pull the plug.

DISCLAIMER: I’ve followed these instructions myself, first on a local Raspberry Pi to test the commands out, and then on a hosted Raspberry Pi. In both cases they’ve worked flawlessly. And not once have i locked myself out of any of the boxes.

That may not always be so! Packages change names, configuration options change, but tutorials rarely follows suit. It’s extremely important that you watch the output of the commands, and check for errors

Anyway, these instructions work for me, and are intended for me. I’m sharing them here hoping that someone might find them useful, but don’t blame me if they blow up your Raspberry Pi!

Now that we’ve established the playground, enjoy, and feel free to contact me on Twitter @jimmyselgen

Changelog

Initial Setup

Change the pi users password

Even though we’re not going to use the pi user, the password should still be changed.

~$ sudo passwd pi

Setting up a new user

Replace default ‘pi’ user with your own user (replace username with your own username)

~$ sudo adduser username

Configuring sudo

Edit the sudoers file to allow your new user to use sudo.

~$ sudo visudo

Find the line that reads

pi ALL=(ALL) NOPASSWD: ALL

and add the following (again, replace username with the user you created above)

username ALL=(ALL) ALL

Save the file, and in a new terminal open a new connection to the Raspberry Pi and verify that you can use sudo as your new user. If everything works you’re ready to remove the Pi user from the sudoers file.

Comment out the pi user from sudoer’s

~$ sudo visudo

And change the ‘pi’ line to

#pi ALL=(ALL) NOPASSWD: ALL

In theory you can now delete the pi user, but since we’ll be restricting logins to specified users only, and use SSH key authentication, i don’t see the need for it.

Setting up public key authentication

Generate a SSH key

You can skip this step if you already have a SSH key you use for remote logins

Generate a SSH key for authorization on your local machine

~$ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/home/user/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/user/.ssh/id_rsa.
Your public key has been saved in /home/user/.ssh/id_rsa.pub.
The key fingerprint is:
22:25:25:a6:3b:72:cb:e4:37:ea:46:88:70:e5:fc:10 user@vali
The key's randomart image is:
+--[ RSA 2048]----+
|    o .          |
|   oEo           |
|  .+...          |
|. ..+o           |
|+o= .o. S        |
|o*.o ...         |
| .+ o            |
|  .o .           |
| oo              |
+-----------------+

Copy SSH public key to the Raspberry Pi

Login as the new user, and upload your public key. ~$mkdir ~/.ssh

Assuming you’re on a unix machine, from your local machine (replace username with the user your created above, and raspberrypi.pcextreme.nl with your Raspberry Pi’s hostname or IP address)

~$scp ~/.ssh/id_rsa.pub username@raspberrypi.pcextreme.nl:.ssh/authorized_keys

Disable password logins in sshd

Edit the ssh server config

~$ sudo vi /etc/ssh/sshd_config

and change PasswordAuthentication to ’no’. Remember to enable the line

# Change to no to disable tunnelled clear text passwords
PasswordAuthentication no

Whitelisting SSH users

Next we’ll setup a list of users allowed to SSH into the Raspberry Pi.

The easiest way to maintain a list of users is the group file. So edit the groups file

~$ sudo vigr

and find the line reading

ssh:x:103:

and append your username right after the :

ssh:x:103:username

After you save the file, you’ll need to update the shadow group file as well

~$ sudo vigr -s

And make the same changes (add your username to the ssh group)

Next we’re going to edit the SSH Server config. Edit /etc/ssh/sshd_config and add the following right at the bottom

AllowGroups ssh

Save the file and exit the editor. Before proceeding we need to test the new setup, since this could potentially lock you out of your Raspberry Pi. To test, enter the following

~$ sudo sshd -t

If the command just exits, without any output, everything should be fine. Now we need to restart the SSH server.

~$ sudo /etc/init.d/sshd restart

This will spawn a new SSH server, leaving your current session running. Next try to make a new SSH connection to your Raspberry Pi.

Hardware Watchdog

The Raspberry Pi has a hardware watchdog on-chip. A hardware watchdog is basically just a hardware timer register that needs to be “reset” every so often, or it will trigger a reboot.

To enable the watchdog, load the bcm2708_wdog module

~$ sudo modprobe bcm2708_wdog
~$ sudo sh -c 'echo "bcm2708_wdog" >> /etc/modules'

This will install the watchdog hardware driver, but by itself it doesn’t do anything. To handle faults/hangs/crashes we need a piece of software to feed the watchdog, and Linux has just the thing.

The watchdog daemon is a standard daemon that watches your system for various parameters, and “stops feeding the dog” if something is amiss. Among the watched parameters are :

  • Is the process table full?
  • Is there enough free memory?
  • Is the average work load too high?
  • Has a file table overflow occurred?

If any of the above tests fail, the watchdog daemon will “stop feeding the dog”, and a reboot will occur shortly after (60 seconds or so)

To install the watchdog daemon

~$ sudo apt-get install watchdog

When it finishes installing edit the file /etc/watchdog.conf and uncomment the line saying “watchdog_device”

watchdog-device = /dev/watchdog

To enable monitoring for when the Raspberry Pi is hung, i.e. a forkbomb, uncomment the lines for monitoring load

max-load-1      = 24
max-load-5      = 18
#max-load-15    = 12

This will make the watchdog daemon reboot your Raspberry Pi whenever the system load reaches the given limit. In the above example, whenever we reach a cpu load of 24 (meaning your Raspberry Pi has 24 active processes waiting to run), or a 5 minute average of 18.

Add the following to the bottom of the file to enable monitoring for checking if sshd is running.

pidfile   = /var/run/sshd.pid

Then enable the daemon

~$ sudo update-rc.d watchdog defaults
~$ sudo service watchdog start

Your Raspberry Pi should now reboot automatically whenever it hangs, crashes, or sshd dies.

WARNING I’ve experimented with various watchdog settings, and both “ping” and “interface” seems to be bugged on the Raspberry Pi, causing the Raspberry Pi to go into a reboot loop. My initial idea was to enable monitoring of eth0 and a ping monitor of my default gateway to reboot the Raspberry Pi whenever network connectivity disappeared, but, for now at least, that doesn’t seem to work. If at all possible, test on a local Raspberry Pi before experimenting with those settings.

Security (or locking down your Raspberry Pi)

Firewall

Safety First!

A good idea when testing firewall rules on a remote machine, is to schedule a reboot of the machine in 5-10 minutes, before loading the new rules. That way, if the script locks you out, your machine will reboot by itself and the bad rules will be flushed.

Schedule a reboot by entering the following command

~$ sudo nohup shutdown -r dd:mm &

Where dd:mm is replaced by a timestamp in the future, e.g. 12:00

To cancel the reboot (if all went well), use

~$ sudo shutdown -c	

Rules

While there are easier options, like shorewall, I prefer to maintain my own iptables script. I’ve copied it in full here, but expect to edit to your own needs

    #/usr/bin/env bash
    #Prevent lockout
    iptables -P INPUT ACCEPT
    iptables -P OUTPUT ACCEPT

    iptables -F
    
    #Allow loopback connections
    iptables -A INPUT -i lo -j ACCEPT
    
    #Allow established connections
    iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
    
    #Drop fast connects most likely bot
    iptables -A INPUT -p tcp -i eth0 -m state --state NEW -m recent --set
    iptables -A INPUT -p tcp -i eth0 -m state --state NEW -m recent --update --seconds 30 --hitcount 10 -j DROP
    
    #Allow SSH
    iptables -A INPUT -p tcp --dport ssh -j ACCEPT
    iptables -A OUTPUT -p tcp --sport ssh -j ACCEPT
    
    #Allow http
    iptables -A INPUT -p tcp --dport http -j ACCEPT
    iptables -A OUTPUT -p tcp --sport http -j ACCEPT
    
    #Allow https
    iptables -A INPUT -p tcp --dport https -j ACCEPT
    iptables -A OUTPUT -p tcp --sport https -j ACCEPT
    
    #Silently DROP broadcasts
    iptables -A INPUT -i eth0 -d 255.255.255.255 -j DROP
    
    #Log dropped packets
    iptables -A INPUT -m limit --limit 2/min -j LOG --log-prefix "DROP: "
    
    #Drop anything else
    iptables -A INPUT -j DROP
    
    #Allow output
    iptables -A OUTPUT -j ACCEPT
    
    #Change policy
    iptables -P INPUT DROP
    iptables -P FORWARD DROP
    iptables -P OUTPUT DROP
    
    #IPv6 rules
    
    #Prevent lockout 
    ip6tables -P INPUT ACCEPT
    ip6tables -P OUTPUT ACCEPT
    
    #Flush old rules
    ip6tables -F
    
    #Allow connections on loopback interface
    ip6tables -A INPUT -i lo -j ACCEPT
    
    #Allow established connections
    ip6tables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
    
    #Allow SSH
    ip6tables -A INPUT -p tcp --dport ssh -j ACCEPT
    ip6tables -A OUTPUT -p tcp --sport ssh -j ACCEPT
    
    #Allow http
    ip6tables -A INPUT -p tcp --dport http -j ACCEPT
    ip6tables -A OUTPUT -p tcp --sport http -j ACCEPT
    
    #Allow https
    ip6tables -A INPUT -p tcp --dport https -j ACCEPT
    ip6tables -A OUTPUT -p tcp --sport https -j ACCEPT
    
    #Allow ICMPv6
    ip6tables -A INPUT -p icmpv6 --icmpv6-type destination-unreachable -j ACCEPT
    ip6tables -A INPUT -p icmpv6 --icmpv6-type packet-too-big -j ACCEPT
    ip6tables -A INPUT -p icmpv6 --icmpv6-type time-exceeded -j ACCEPT
    ip6tables -A INPUT -p icmpv6 --icmpv6-type parameter-problem -j ACCEPT

    # Allow some other types in the INPUT chain, but rate limit.
    ip6tables -A INPUT -p icmpv6 --icmpv6-type echo-request -m limit --limit 900/min -j ACCEPT
    ip6tables -A INPUT -p icmpv6 --icmpv6-type echo-reply -m limit --limit 900/min -j ACCEPT

    # Allow others ICMPv6 types but only if the hop limit field is 255.
    ip6tables -A INPUT -p icmpv6 --icmpv6-type router-advertisement -m hl --hl-eq 255 -j ACCEPT
    ip6tables -A INPUT -p icmpv6 --icmpv6-type neighbor-solicitation -m hl --hl-eq 255 -j ACCEPT
    ip6tables -A INPUT -p icmpv6 --icmpv6-type neighbor-advertisement -m hl --hl-eq 255 -j ACCEPT
    ip6tables -A INPUT -p icmpv6 --icmpv6-type redirect -m hl --hl-eq 255 -j ACCEPT

    #Log dropped packets
    ip6tables -A INPUT -m limit --limit 2/min -j LOG --log-prefix "DROP: "
    
    #Drop everything else
    ip6tables -A INPUT -j DROP
    
    #Allow output
    ip6tables -A OUTPUT -j ACCEPT 
    
    #Change policy to deny everything by default
    ip6tables -P INPUT DROP
    ip6tables -P FORWARD DROP
    ip6tables -P OUTPUT DROP

Save the script as pi_firewall.sh, set execute permissions on it and run it.

~$chmod +X pi_firewall.sh
~$ sudo ./pi_firewall.sh

Enable firewall at boot

If everything works, save your firewall

~$ sudo sh -c "iptables-save > /etc/iptables.rules"
~$ sudo sh -c "ip6tables-save > /etc/ip6tables.rules"

add a pre-up command to /etc/network/interfaces

iface eth0 inet static
    address 92.63.xxx.xxx
    netmask 255.255.255.0
    gateway 92.63.xxx.xxx

right below gateway, like this

iface eth0 inet static
    address 92.63.xxx.xxx
    netmask 255.255.255.0
    gateway 92.63.xxx.xxx
pre-up iptables-restore < /etc/iptables.rules
pre-up ip6tables-restore < /etc/ip6tables.rules

Log checking

Ideally you should be checking your servers logs for problems, attacks, etc every day, which is where logcheck comes in. Logcheck is a small daemon that scans your logfiles for problems every 15 minutes, and emails you a summary of whatever problems might have arisen.

From the Logcheck wiki

Logcheck is a simple utility which is designed to allow a system administrator to view the logfiles which are pro>duced upon hosts under their control.

It does this by mailing summaries of the logfiles to them, after first filtering out “normal” entries.

Normal entries are entries which match one of the many included regular expression files contain in the database.

To install, simply execute

~$ sudo apt-get install logcheck

By default logcheck runs at 2 mins past every hour, which in turn generates 24 emails per day. If you want to change the interval at which logcheck runs, edit /etc/cron.d/logcheck and find the following line

2 * * * *       logcheck    if [ -x /usr/sbin/logcheck ]; then nice -n10 /usr/sbin/logcheck; fi 

Change the execution interval according to the specification for cron. I.e. if you would like logcheck to run at 08:02 every day, you would change it to

2 8 * * *       logcheck    if [ -x /usr/sbin/logcheck ]; then nice -n10 /usr/sbin/logcheck; fi 

Avoid log spam from iptables

With the firewall rules we setup before, every dropped packet goes into /var/log/messages, which will trigger logcheck to alert you that something suspicious has occurred. If you want to know everything that happens on your box, feel free to skip this section.

To avoid triggering logcheck, we will redirect iptables log entries to /var/log/iptables.log

~$ sudo vi /etc/rsyslog.d/iptables.conf

Add the following lines

:msg,contains,"DROP:" -/var/log/iptables.log
& ~

The first line means “send everything that contains “DROP:” to /var/log/iptables.log and the second line just tells rsyslog to discard the line that was matched in the previous rule. The ‘-’ before the logfile tells rsyslog that we don’t want to synchronize the logfile every time we write to it. This may cause a dataloss if the system crashes right after writing to the file, but it’ll save some CPU power, especially on verbose loggers. Additionally it will save a bit on wear & tear on the SD Card.

Save the file, and restart the syslog service.

~$ sudo service rsyslog restart

You can verify that you now have a file name /var/log/iptables.log

➜  ~  ls -l /var/log/iptables.log
-rw-r----- 1 root adm 0 Jul 18 22:29 /var/log/iptables.log

To avoid filling up your SD-Card we need to tell logrotate about the new logfile. Edit /etc/logrotate.d/rsyslog and add a line for your iptables.log file. I chose to rotate my iptables log file daily, so i added it above /var/log/syslog

/var/log/iptables.log
/var/log/syslog
{
    rotate 7
    daily
    missingok
    notifempty
    delaycompress
    compress
    postrotate
    invoke-rc.d rsyslog rotate > /dev/null
    endscript
}

This will keep a backlog of your iptables.log files for 7 days, gzip compressed, named from /var/log/iptables.log.1, to /var/log/iptables.log.7.gz

Preventing attacks

Fail2ban

Fail2ban is small daemon that monitors your logfiles for failed login attempts, and when it finds a pattern matching its configuration, it automatically adds a iptables rule that bans the offending source IP address for a specified period of time.

From Wikipedia

Fail2ban operates by monitoring log files (e.g. /var/log/pwdfail, /var/log/auth.log, etc.) for selected entries and running scripts based on them. Most commonly this is used to block selected IP addresses that may belong to hosts that are trying to breach the system’s security. It can ban any host IP that makes too many login attempts or performs any other unwanted action within a time frame defined by the administrator.[2] Fail2ban is typically set up to unban a blocked host within a certain period, so as to not “lock out” any genuine connections that may have been temporarily misconfigured.[3] However, an unban time of several minutes is usually enough to stop a network connection being flooded by malicious connections, as well as reducing the likelihood of a successful dictionary attack.

To install, simply execute

~$ sudo apt-get install fail2ban

psad

psad is another daemon (3 daemons actually) that monitors your logfile for potential threads. I’ve replaced the previous section on Portsentry with psad because psad performs better (IMO), and works well with Fail2ban.

From the psad website

psad is a collection of three lightweight system daemons (two main daemons and one helper daemon) that run on Linux machines and analyze iptables log messages to detect port scans and other suspicious traffic. A typical deployment is to run psad on the iptables firewall where it has the fastest access to log data.

To install, run the following commands

~$ sudo apt-get install psad

Then open up /etc/psad/psad.conf and change the hostname

### Machine hostname
HOSTNAME                    my.hostname.com;

If you moved the iptables log to /var/log/iptables.log edit the location of the file that psad searches for iptables logs

#IPT_SYSLOG_FILE             /var/log/messages;
IPT_SYSLOG_FILE             /var/log/iptables.log;

If you have any hosts that you never want to be banned (hint: your local workstation, NFS servers), put them in /etc/psad/auto_dl

If, like me, your workstation is on a DHCP assigned IP address, you might want to ignore a larger subnet. To find which subnet you’re on, find your IP address, and look it up using whois

~$whois xxx.xxx.xxx.xxx
% This is the RIPE Database query service.
% The objects are in RPSL format.
%
% The RIPE Database is subject to Terms and Conditions.
% See http://www.ripe.net/db/support/db-terms-conditions.pdf

% Note: this output has been filtered.
%       To receive output for a database update, use the "-B" flag.

% Information related to '178.157.192.0 - 178.157.223.255'

% Abuse contact for '178.157.192.0 - 178.157.223.255' is 'abuse@energimidt.dk'

inetnum:        178.157.192.0 - 178.157.223.255
netname:        FBB_RESIDENTIAL_DHCP_DYNAMIC
descr:          Infrastructure EM - DHCP assignments residential users
remarks:        INFRA-AW
country:        DK
admin-c:        ECR4-RIPE
tech-c:         ECR4-RIPE
status:         ASSIGNED PA
mnt-by:         EM-MNT
mnt-lower:      EM-MNT
mnt-routes:     EM-MNT
source:         RIPE # Filtered

role:           EM Contact Role
address:        Tietgensvej 2-4, 8600 Silkeborg, DK
admin-c:        ARJ7-RIPE
admin-c:        SJ2277-RIPE
tech-c:         ARJ7-RIPE
tech-c:         SJ2277-RIPE
abuse-mailbox:  abuse@energimidt.dk
nic-hdl:        ECR4-RIPE
mnt-by:         EM-MNT
source:         RIPE # Filtered

% Information related to '178.157.192.0/18AS43557'

route:          178.157.192.0/18
descr:          EnergiMidt Route
origin:         AS43557
remarks:        Abuse issues should be reported to abuse@energimidt.dk
mnt-by:         EM-MNT
mnt-routes:     EM-MNT
source:         RIPE # Filtered

% This query was served by the RIPE Database Query Service version 1.66.3 (WHOIS3)

We’re looking for something like route: 178.157.192.0/18. Just put your route information a the bottom of /etc/psad/auto_dl

178.157.192.0/18        0;

Again, a good idea when testing stuff like this is to schedule a reboot at some point in the future. Finally, restart the psad daemon

~$ sudo service psad restart

Owncloud

I followed this tutorial with the various changes mentioned below (btsync mainly)

I chose nginx as my server since i feel it performs better, and tests seems to back this up.

BTSync

BTSync synchronization configuration

Because this is a remote server, and i don’t expect to setup synchronized drives other than initial setup, i prefer not to have my Raspberry Pi listen on the port for the BTSync administration interface.

Instead i forward a local port to the machine via ssh, like this (replace raspberrypi.pcextreme.nl with the hostname/ip address of your raspberry pi)

~$ssh -f raspberrypi.pcextreme.nl -L 8888:localhost:8888 -N

Now you can just connect to http://localhost:8888 and configure your BTSync directories.

BTSync launched from /etc/init.d

Since this is a server, and i prefer not to “waste” RAM by launching X, i chose to launch btsync from /etc/init.d instead. To do so i used this script. Instructions for setting it up are on the page, below the script.

Monitoring

Generating daily reports

I use this script for sending me daily status reports from the server.

Save the script to /usr/local/bin/daily_stats.pl

~$ sudo chmod 700 /usr/local/bin/daily_stats.pl
~$ sudo apt-get install sendemail libcrypt-ssleay-perl libio-socket-ssl-perl

then run

~$ sudo crontab -e

and add the following line

00 12 * * * /usr/local/bin/daily_stats.pl

Save the file and exit from the editor.

Sending reports via GMail

Before you setup your mail server, you must set your Reverse DNS. If GMail cannot resolve your servers IP address to the hostname you claim to be sending from, then GMail will simply refuse to deliver mail from you.

You can set your reverse DNS on your Raspberry Pi here1

My preferred MTA is Postfix, to install it

~$ sudo apt-get install postfix

Enable aliases

~$ sudo postconf -e "alias_maps = hash:/etc/aliases"

Then edit /etc/aliases2

root: username
username: user.name@gmail.com

When you’re done editing /etc/aliases, you need to tell Postfix to reload the alias file

~$ sudo newaliases

The Debian Wiki Postfix page has more configuration options.

And you’re done. All mail sent to your local user on the Raspberry Pi should now get forwarded to your gmail account.

Staying up to Date

Finally done, and hopefully everything is setup and working perfectly. Now let us make sure it stays that way.

Setting up automatic updates

Raspbian (debian) has a little tool that will download fresh apt sources every night, and email you if some of your packages need upgrading.

~$ sudo apt-get install cron-apt

With the default configuration, cron-apt will run every night at 04:00, and send a notification to root, and assuming you’ve setup your MTA like above, all mails for root will get forwarded to your account.

Cron-apt doesn’t install anything by default, so you still need to do that manually.

Conclusion

Congratulations.

You now have a relatively secure Raspberry Pi running with BTSync and OwnCloud. Provided you’ve selected good passwords, the only remaining attack vector will be bugs in the software you’ve installed, so remember to login and update your software every time cron-apt sends you an email.

Further Reading

Here is a list of resources i’ve used while putting together this little guide.

Changelog

  • 2013/07/10 - Initial version.
  • 2013/07/11
    • Updated with psad configuration.
      • Added logging rules to firewall script.
      • Added section on psad configuration.
      • Removed Portsentry section.
  • 2013/07/12
    • Reordered firewall script.
      • Moved fast connection limit before services.
      • Silently drop broadcasts.
    • Added section on changing logcheck execution interval
  • 2013/07/15
    • Updated firewall script to allow some icmpv6 traffic.
  • 2013/07/16
    • Updated watchdog section to include monitoring of sshd.
  • 2013/07/18
    • Added section on iptables logging to a seperate file.
  • 2013/07/22
    • Moved changelog to bottom of document.
  • 2014/01/06
    • Updated section on watchdog installation due to conflict with wolfram-engine
  • 2017/05/24
    • Removed part about hardening sudo. This was more security by obscurity and added no real security value.
  • 2017/08/14
    • Removed part about Wolfram Engine conflicting with watchdog. It no longer conflicts on latest Raspbian.

  1. If you’re setting up a server that has no reverse DNS, this might come in handy. ↩︎

  2. Alternatively you can setup a forward address in ~/.forward, just enter your email address. ↩︎


See also