How I’m (slowly) moving off the cloud with Nextcloud

How I’m (slowly) moving off the cloud with NextcloudA walkthrough on how I made my Raspberry Pi a decent self-hosted cloud.

Rohit KapurBlockedUnblockFollowFollowingJan 31I bought a Raspberry Pi from Amazon with the goal of moving off of Google Photos to a self-hosted photo backup solution that’s nearly as seamless.

My goals for this project:A Nextcloud instance on my internal network which could backup photos from my phoneA VPN to access the server from outside my network with Dynamic DNSAn internal DNS server to allow me to have an internal subdomainSSL/TLS certificate from LetsEncrypt for the internal domainThere was one thing I had in mind when coming up with these goals: I didn’t just want my personal cloud to be private, I wanted it to be convenient.

My setupFirst off, here are the things I needed for my setup:Raspberry Pi B+Samsung 128GB MicroSD cardMicroUSB Power SupplyPlastic CasePowered USB HubWestern Digital 2TB External Hard DriveEthernet CableA domain nameThe most important thing here is a proper power supply.

Most phone chargers will work to power the Pi initially, but they’ll fail when they run something that requires additional power.

To start off, I installed Raspbian Stretch Lite on my Raspberry Pi and connected to it via SSH.

To avoid duplicating too much of what’s already available online, you can just follow the guide here.

Once I installed the operating system and was able to SSH into it, I ran the following commands to install updates:sudo apt-get updatesudo apt-get upgradeAnd then changed the default password.

passwdSetting up Dynamic DNSMost ISPs will periodically change your external IP address.

If you want to be able to reliably access your home network through a VPN, you’ll need to set up Dynamic DNS.

Dynamic DNS will allow you to assign a subdomain to your home network and automatically update the assigned IP address if it changes.

Depending on your domain registrar, the following steps will be different, but I use Google Domains so I’ll outline the process for that here.

First, navigate to the DNS section and under ‘Synthetic Records’, select ‘Dynamic DNS’ from the dropdown.

Next, enter a subdomain and click ‘Add’.

Make note of the username and password that is generated by Google Domains.

Once you’ve set up your DNS, the next step is to install the ddclient package.

sudo apt-get install ddclientYou can select the default options for each prompt, we’ll be reconfiguring this information in the next step.

For username and password, paste in the information you got from Google Domains in the previous step.

Now, open up the ddclient configuration file:sudo nano /etc/ddclient.

confNext, replace the content of the file with the following:protocol=dyndns2use=webserver=domains.

google.

comssl=yeslogin=generated_usernamepassword='generated_password'test.

example.

comReplace generated_username and generated_password with the username and password you copied down earlier.

Also, replace test.

example.

com with the subdomain you set up.

CTRL+X to exit and Y and Enter to save.

Test your configuration and make sure at least one of the lines starting with use=web has your public IP address.

sudo ddclient -queryIt should contain a line that looks similar to this:use=web, web=loopia address is 67.

122.

333.

444Next, we want to make sure the service is running periodically.

Open the following file:sudo nano /etc/default/ddclientChange run_daemon="false" to run_daemon="true" if it isn’t set already.

CTRL+X to exit and Y and Enter to save.

If everything goes well, start the ddclient service:sudo systemctl start ddclientLastly, we want to add a cron to ensure that the ddclient service is updating.

We’ll create a blank file with a cronjob to force ddclient to update:sudo nano /etc/cron.

daily/ddclientAdd this to the empty file:#!/bin/sh/usr/sbin/ddclient -forceMake it executable:sudo chmod +x /etc/cron.

daily/ddclientAnd that’s it for Dynamic DNS.

Setting a Static IPFor most of the following sections, you’ll need to make sure your Pi maintains the same IP address consistently.

For this, you need to set up Static IP or a DHCP reservation.

> Important: If your router has the capability to do a DHCP reservation, do that, it’s much easier.

Because every router is different, this is something you’ll have to find online.

If you can’t do a DHCP reservation, you’ll need to set a Static IP on the Pi itself.

To start, open up the dhcpcd.

conf file.

sudo nano /etc/dhcpcd.

confNext, add the following to the end of the file.

You’ll need to replace both instances of 192.

168.

1.

1 with your router’s IP address and replace 192.

168.

1.

2 with the IP address you want to set for your Pi.

interface eth0static ip_address=192.

168.

1.

2/24static routers=192.

168.

1.

1static domain_name_servers=192.

168.

1.

1Also, if you’re using Wi-Fi instead of a wired connection, replace eth0 with wlan0.

Finally, reboot your Pi.

sudo rebootSetting up Internal DNSAn internal DNS server is what will allow you to set a domain name with a LetsEncrypt SSL/TLS on your NextCloud instance.

Technically, you could put an internal IP on your external DNS, but that would reveal information about your internal network and it’s best to avoid that where possible.

To start off, we need to install dnsmasq.

sudo apt-get install dnsmasqOpen up the configuration file.

sudo nano /etc/dnsmasq.

confYou should see a bunch of options that are commented out.

Scroll down to the bottom and paste in the following:no-hostsaddn-hosts=/etc/hosts.

dnsmasqcache-size=150no-negcacheno-hosts tells dnsmasq to ignore your hosts fileaddn-hosts tells it to look for a different hosts file (we’ll set this up in the next step)cache-size option sets how many names to be cached (you can increase this if you want)no-negcache avoids caching items that return “no such domain” responses from upstream serversNext, let’s create our hosts.

dnsmasq file.

sudo nano /etc/hosts.

dnsmasqIn this file, we need to add any hosts that we don’t want being resolved by the upstream DNS (in my case Cloudflare).

Your hosts file should look similar to this:192.

168.

1.

2 test.

example.

com192.

168.

1.

2 nextcloud.

example.

comReplace all instances of 192.

168.

1.

2 with your Pi’s IP address and replace example.

com with your own domain.

In addition, replace test.

example.

com with the Dynamic DNS domain you set up earlier.

This will allow you connect to your Pi using the same domain as you use externally.

CTRL+X to exit and Y and Enter to save.

Restart dnsmasq to load the changes.

sudo systemctl restart dnsmasqsudo systemctl enable dnsmasqNext, you need to make sure your devices are using the Pi as their DNS server.

I’ve included the steps for macOS and iOS below.

For macOS,Open System Preferences and click on NetworkOn the left, click the interface you’re connected to (usually Wi-Fi or Ethernet)Click on Advanced in the bottom-right cornerSelect DNS from the top navigation barUnder DNS Servers on the left, click the + and add your Pi’s IP address to the list of DNS servers your computer should use.

For iOS,Open the Settings app and select Wi-FiTap on the ⓘ next to the name of your routerTap Configure DNS and select ManualUnder DNS Servers tap Add Server and enter your Pi’s IP addressFor other devices, you should be able to find guides online, but here are sources for Android, Windows and Ubuntu.

Now, if you type nslookup test.

example.

com from your computer, you should see it return your Pi’s IP address.

$ nslookup test.

example.

comServer: 192.

168.

1.

2Address: 192.

168.

1.

2#53Name: test.

example.

comAddress: 192.

168.

1.

2Setting up OpenVPNWhile it was possible to do this manually, I decided to use a simpler setup script created by Stanislas Lange.

Tell him thanks by clicking here!First off, you should cd into your home directory.

cd ~Next, download the script by running:curl -O https://raw.

githubusercontent.

com/Angristan/openvpn-install/master/openvpn-install.

shMake the script executable.

chmod +x openvpn-install.

shNow, run the script.

sudo .

/openvpn-install.

shThe script will ask for your internal IP address.

Most of the time this is pre-filled.

IP address: 192.

168.

1.

2Next, it will ask for your Public IPv4 address or hostname.

This will be the hostname for the dynamic DNS we setup earlier.

Public IPv4 address or hostname: test.

example.

comFor IPv6 support, leave the default n.

Do you want to enable IPv6 support (NAT)? [y/n]: nFor the next question, I chose a random port instead of leaving the default 1194.

It’s not necessary to choose a random one, but it gives you a slightly better chance against port scanners that scan the internet for common network services.

What port do you want OpenVPN to listen to? 1) Default: 1194 2) Custom 3) Random [49152-65535]Port choice [1-3]: 3Random Port: 59348If you didn’t pick the default, note down the port number the script assigned.

In my case, it is 59348.

For the next question, leave the default configuration:What protocol do you want OpenVPN to use?UDP is faster.

Unless it is not available, you shouldn't use TCP.

1) UDP 2) TCPProtocol [1-2]: 1Next, you can choose any DNS provider you feel most comfortable with, but I used Cloudflare for their focus on privacy.

What DNS resolvers do you want to use with the VPN? 1) Current system resolvers (from /etc/resolv.

conf) 2) Self-hosted DNS Resolver (Unbound) 3) Cloudflare (Anycast: worldwide) 4) Quad9 (Anycast: worldwide) 5) Quad9 uncensored (Anycast: worldwide) 6) FDN (France) 7) DNS.

WATCH (Germany) 8) OpenDNS (Anycast: worldwide) 9) Google (Anycast: worldwide) 10) Yandex Basic (Russia) 11) AdGuard DNS (Russia)DNS [1-10]: 3I left the next two options as their default:Do you want to use compression?.It is not recommended since the VORACLE attack make use of it.

Enable compression?.[y/n]: nDo you want to customize encryption settings?Customize encryption settings?.[y/n]: nAfter the script installs the required packages and dependencies, it will ask you for a client name.

This is the name of the first user you want to add to your VPN server.

Client name: vpnuserThe script will also ask you if you want to add a password to the client.

This allows you to encrypt the private key generated with a password.

This isn’t required, but there’s no harm in encrypting your key since most clients will let you save the password and you won’t have to enter it repeatedly.

Do you want to protect the configuration file with a password?(e.

g.

encrypt the private key with a password) 1) Add a passwordless client 2) Use a password for the clientSelect an option [1-2]: 2Enter PEM pass phrase:Verifying – Enter PEM pass phrase:Next you need to install the OpenVPN app for iOS or Android on your device and copy over the certificate.

Once you have the app downloaded, enter the following command on your Pi to start a simple web server you can access from your phone:python -m "SimpleHTTPServer"Now, on your phone, open your web browser and navigate to http://192.

168.

1.

2:8000 (replace 192.

168.

1.

2 with the IP address of your Pi) and download the .

ovpn file and add the profile to your OpenVPN app.

CTRL+C out of the Python web server once you’ve transferred the file.

Open the OpenVPN configuration file.

sudo nano /etc/openvpn/server.

confAdd the following above the line starting with push "dhcp-option(Replace the IP address with your Pi’s IP address).

push "dhcp-option DNS 192.

168.

1.

2"Restart OpenVPN.

sudo systemctl restart openvpnThe last step is to set up port forwarding on your router and forward the OpenVPN port you noted down earlier to the local IP address of your Pi.

Because everyone has a different router, I’ll just link a generic guide here.

Once port forwarding is set up, you should be able to connect to your VPN.

Setting up the web serverThe first step is to install Apache and enable it to start on boot:sudo apt-get install apache2sudo systemctl enable apache2Next, we need to install MySQL (MariaDB) and enable it to start at boot:sudo apt-get install mariadb-serversudo systemctl enable mariadbTo help secure your MySQL instance, run the following and answer Y to the prompts.

This will help you set a root password and remove unnecessary tables from your server.

sudo mysql_secure_installationYour terminal should look a little bit like this:Enter current password for root (enter for none):Set root password?.[Y/n] YNew password:Re-enter new password:Remove anonymous users?.[Y/n] YDisallow root login remotely?.[Y/n] YRemove test database and access to it?.[Y/n] YReload privilege tables now?.[Y/n] YNext, you need to install PHP:sudo apt-get install php7.

0 libapache2-mod-php7.

0You’ll also need some additional PHP packages:sudo apt-get install php7.

0-common php7.

0-gd php7.

0-json php7.

0-mysql php7.

0-curl php7.

0-mbstring php7.

0-intl php-imagick php7.

0-xml php7.

0-zip php7.

0-mysqlSetting up LetsEncryptLetsEncrypt is a free certificate authority that’ll let you have SSL/TLS certificate for your Nextcloud instance.

First, move into your home directory and download the latest version of Certbot, an automated certificate setup tool, to your Pi.

cd ~sudo wget https://dl.

eff.

org/certbot-autoMake the script executable.

sudo chmod a+x certbot-autoNext, run the following (replace example.

com with your own domain).

sudo .

/certbot-auto certonly –manual –preferred-challenges dns-01 -i apache -d "*.

example.

com" –server https://acme-v02.

api.

letsencrypt.

org/directoryYou may be asked to install some additional dependencies.

Enter Y if prompted.

(This may take a while).

First, enter the email address you’d like to receive important information at:Enter email address (used for urgent renewal and security notices) (Enter 'c' to cancel): email@example.

comYou’ll also need to read and agree to the terms of service:Please read the Terms of Service athttps://letsencrypt.

org/documents/LE-SA-v1.

2-November-15-2017.

pdf.

You must agree in order to register with the ACME server athttps://acme-v02.

api.

letsencrypt.

org/directory- – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – (A)gree/(C)ancel: AThe next one is up to you.

Would you be willing to share your email address with the Electronic Frontier Foundation, a founding partner of the Let's Encrypt project and the non-profit organization that develops Certbot?.We'd like to send you email about our work encrypting the web, EFF news, campaigns, and ways to support digital freedom.

– – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – (Y)es/(N)o: NTo obtain a certificate, you must agree to the IP logging.

NOTE: The IP of this machine will be publicly logged as having requested this certificate.

If you're running certbot in manual mode on a machine that is not your server, please ensure you're okay with that.

Are you OK with your IP being logged?- – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – (Y)es/(N)o: YNext, you need to add a TXT record to your external DNS.

This allows LetsEncrypt to verify that you actually own the domain you’re trying to obtain a certificate for.

Please deploy a DNS TXT record under the name_acme-challenge.

example.

com with the following value:KYL9wfbBTif3bfdMZ7xwwqxWgXsvLVNPePmuxGEmnMYBefore continuing, verify the record is deployed.

– – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – Press Enter to Continue> Important: Before pressing Enter, take the value displayed by certbot and add it to your external DNS.

I’ll show how to do that for Google Domains, but the process should be relatively similar for most registrars.

Open the DNS section for Google Domains.

Scroll down to ‘Custom resource records’ and and select TXT from the dropdown.

Next add _acme-challenge to the name field on the left and in the data field, paste the value from the certbot script.

Click ‘Add’ once you’re done.

> Important: Wait for a few minutes for the DNS to propagate.

If you don’t wait long enough, the script will fail and you’ll have to start this process from the beginning.

Now that you’ve waited, you can press Enter and continue the script.

Waiting for verification.

Cleaning up challengesIMPORTANT NOTES: – Congratulations!.Your certificate and chain have been saved at: /etc/letsencrypt/live/example.

com/fullchain.

pem Your key file has been saved at: /etc/letsencrypt/live/example.

com/privkey.

pem Your cert will expire on 2019-04-30.

To obtain a new or tweaked version of this certificate in the future, simply run certbot- auto again.

To non-interactively renew *all* of your certificates, run "certbot-auto renew" – If you like Certbot, please consider supporting our work by:Donating to ISRG / Let's Encrypt: https://letsencrypt.

org/donateDonating to EFF: https://eff.

org/donate-leNow that we’ve downloaded the certificates, it’s time to link them to Nextcloud.

Installing NextcloudFirst, we need to create a database user for the Nextcloud instance.

Enter your MySQL server and enter the root password you created in the previous section:sudo mariadb -u root -pRun the following commands to create a new database user.

Be sure to change ‘password’ to something secure!MariaDB [(none)]> CREATE DATABASE nextcloud;MariaDB [(none)]> GRANT ALL PRIVILEGES ON nextcloud.

* TO 'nextcloud'@'localhost' IDENTIFIED BY 'your-mysql-password';MariaDB [(none)]> FLUSH PRIVILEGES;MariaDB [(none)]> exit;Now it’s time to download Nextcloud.

Head over to https://download.

nextcloud.

com/server/releases/ and copy the url for the latest version of Nextcloud in .

tar.

bz2 format.

For me, this was nextcloud-15.

02.

tar.

bz2.

To do this, just navigate to the link above, right click the file and click ‘Copy Link Address’.

Next, you’ll want to download and extract that to your Pi’s web directory.

Switch to the web directory by entering:cd /var/www/and then run the following to download and extract Nextcloud.

sudo curl https://download.

nextcloud.

com/server/releases/nextcloud-15.

0.

2.

tar.

bz2 | sudo tar -xjNow, we need to change ownership of the nextcloud folder to the web user and move into that directory:sudo chown -R www-data:www-data nextcloudcd nextcloudRun the following command to set up Nextcloud with MySQL using the password you set before.

Again, be sure to change the Nextcloud administrator’s password to something more secure.

sudo -u www-data php occ maintenance:install –database "mysql" –database-name "nextcloud" –database-user "nextcloud" –database-pass "your-mysql-password" –admin-user "admin" –admin-pass "nextcloud-admin-password"You should see this:Nextcloud was successfully installedWe should run the following command to remove the plaintext password we just entered from our bash history:history -d $(history 2 | head -n 1 | awk '{print $1}')Next, we need to set up Apache Virtual Hosts to access our Nextcloud instance.

Start by opening an empty configuration file:sudo nano /etc/apache2/sites-available/nextcloud.

confPaste the following into the file, but replace all instances of example.

com (in bold) with your own domain.

<VirtualHost *:80> ServerAdmin email@example.

com ServerName nextcloud.

example.

com ServerAlias www.

nextcloud.

example.

com DocumentRoot /var/www/nextcloud ErrorLog ${APACHE_LOG_DIR}/error.

log CustomLog ${APACHE_LOG_DIR}/access.

log combined RewriteEngine On RewriteCond %{HTTPS} off RewriteRule ^/?(.

*) https://%{SERVER_NAME}/$1 [R,L]</VirtualHost><IfModule mod_ssl.

c><VirtualHost *:443> ServerAdmin email@example.

com ServerName nextcloud.

example.

com ServerAlias www.

nextcloud.

example.

com DocumentRoot /var/www/nextcloud ErrorLog ${APACHE_LOG_DIR}/error.

log CustomLog ${APACHE_LOG_DIR}/access.

log combinedSSLEngine on SSLCertificateFile /etc/letsencrypt/live/example.

com/cert.

pem SSLCertificateKeyFile /etc/letsencrypt/live/example.

com/privkey.

pem SSLCertificateChainFile /etc/letsencrypt/live/example.

com/chain.

pem</VirtualHost></IfModule>SSLProtocol all -SSLv2 -SSLv3SSLCipherSuite ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECD$SSLHonorCipherOrder onSSLCompression offThere’s a lot in here, but essentially, this configuration file tells Apache to:look for the site in the /var/www/nextcloud folderredirect http traffic to httpsuse the SSL/TLS certificates located in /etc/letsencrypt/live/example.

comdisable weak SSL ciphers such as SSLv2 and SSLv3.

CTRL+X to exit and Y and Enter to save.

Next, we need to enable a couple of apache modules.

sudo a2enmod sslsudo a2enmod rewriteAnd finally enable the site and restart Apache.

sudo a2ensite nextcloud.

confsudo systemctl restart apache2You should now be able to access your Nextcloud instance at nextcloud.

example.

com from any computer on your network that’s connected to your Pi DNS.

You can now log in with the administrator account and create a separate user account for yourself and change any settings you want to change.

Be sure to download the iOS or Android app and set up photo backup!Setting up External Storage (Optional)Now, everything on Nextcloud should be working, but adding external storage makes Nextcloud more usable.

You can store much larger files without having to worry about remaining space on your SD card.

> Important: This will erase all data on your drive.

If you have any important data, please back it up before starting this process.

First, you have to connect and format your drive.

Plug the drive into a powered USB hub and connect the hub to the Pi.

List all the connected disks:sudo fdisk -lYou should see something like this:Disk /dev/sdx: 1.

8 TiB, 2000365289472 bytes, 3906963456 sectorsUnits: sectors of 1 * 512 = 512 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesNow, note down the disk name (mine is /dev/sdx) and replace that in the next command.

It’s important that you get the correct value here because you can accidentally wipe the wrong disk if you don’t.

Run the following to wipe and format your disk (replace /dev/sdx with your drive name):sudo fdisk /dev/sdx You should see the following:Welcome to fdisk (util-linux 2.

29.

2).

Changes will remain in memory only, until you decide to write them.

Be careful before using the write command.

Enter d in the first prompt.

Command (m for help): dSelected partition 1Partition 1 has been deleted.

Next, enter the following (in bold) to add a new partition (note: leave the first sector and last sector prompts blank):Command (m for help): nPartition type p primary (0 primary, 0 extended, 4 free) e extended (container for logical partitions)Select (default p): pPartition number (1-4, default 1): 1First sector (2048-3906963455, default 2048):Last sector, +sectors or +size{K,M,G,T,P} (2048-3906963455, default 3906963455):Created a new partition 1 of type 'Linux' and of size 1.

8 TiB.

Enter w to write the changes to your disk.

Command (m for help): wThe partition table has been altered.

Now, we need to format the disk as ext4.

Run the following and be sure to replace /dev/sdx1 with the drive name we used in the previous commands.

sudo mkfs.

ext4 /dev/sdx1Next, we need to set a location to mount the drive to.

Create folder called external in the media directory.

sudo mkdir /media/externalEnter the following to mount the drive (again replacing /dev/sdx with your drive name) and cd into that directory and create a nextcloud folder.

sudo mount /dev/sdx1 /media/external/cd /media/externalsudo mkdir /media/external/nextcloudLet’s move our Nextcloud data folder into this drive.

We need to move the folder and create a symbolic between the original location and the new one.

sudo mv /var/www/nextcloud/data/ /media/external/nextcloud/sudo ln -s /media/external/nextcloud/data /var/www/nextcloud/dataNext, we want to ensure that the web server can access the data directory.

sudo chown www-data:www-data /media/external/nextcloud/dataLoad up nextcloud.

example.

com to make sure it’s still working.

Finally, we need to make sure the drive is mounted on each boot.

Enter the following to list out your connected drives.

sudo blkidCopy down the UUID for your external drive.

/dev/sdx1: UUID="e4957cf5-0290-aa6c-41b4-b8673f8da3a9" TYPE="ext4"Open up your fstab file.

sudo nano /etc/fstabPaste the following at the bottom of the file.

Be sure to replace the UUID of my drive with yours.

UUID=e4957cf5-0290-aa6c-41b4-b8673f8da3a9 /media/external ext4 defaults,nofail 0 0The UUID makes sure that if you connect your devices in a different order, it will still mount to the same directory.

Reboot your Pi to finish up.

sudo rebootWrapping UpWe’ve added a lot of different services to our Pi, but now, you should be able to connect to your Nextcloud instance from your home or over VPN and backup your photos and other files.

For maintenance, the only things you need to do are renew your LetsEncrypt certificates (every 90 days) and update Nextcloud and your Pi to the latest versions to keep up with security patches.

.

. More details

Leave a Reply