How to create multiple Apache HTTPS virtual hosts

I am currently migrating from a Cisco ASA to a Palo Alto 850, and need to test as many of the rules as possible before deployment. The configuration has been configured on the PA-850 using the Palo Alto Migration tool (which I’ll be covering in a different post), but the ruleset needs to be tested before I swap the cables over. in this post, I’ll cover how to create multiple Apache HTTPS virtual hosts, including configuring non-standard ports. Whilst configuring one-off sites is not hard, I have some slightly more complicated needs:

There is in the region of 80 different virtual hosts
Some use the same IP address
Some IP addresses are in different subnets
Some have different port numbers
All use HTTPS
I ain’t doing it by hand!

This will run on Ubuntu, so if you are following along at home, all you need a basic install and to follow the next bits.

Installing Apache2 and PHP on Ubuntu 16.0.4

We start by installing Apache2 and PHP, an SSH server and enabling SSL in apache:

sudo apt-get install apache2 php libapache2-mod-php
sudo apt-get install ssh
sudo apt-get install vim
sudo a2enmod ssl
sudo service apache2 restart

OK, so far so good. Next, we’ll consider the issue of different subnets.

Creating VLANs in Ubuntu

sudo apt-get install vlan
sudo modprobe 8021q

Next, we need to create the interfaces:

sudo vconfig add enp0s3 5
sudo vconfig add enp0s3 90
sudo su -c 'echo "8021q" >> /etc/modules'

You should see the following:

Added VLAN with VID == 5 to IF -:enp0s3:-
Added VLAN with VID == 90 to IF -:enp0s3:-

Your interface may not be “enp0s3” – so check first using “ifconfig”! Amend it to suit. In the above commands, we have created a VLAN 5 and a VLAN 90 and made the 8021q module persist on reboots.

Next, we’ll need some interfaces added in /etc/network/interfaces (sudo vim /etc/network/interfaces):

auto enp0s3.5
auto enp0s3.90

iface enp0s3.5 inet static
 address 192.168.5.1
 subnet 255.255.255.0
 network 192.168.5.0
 vlan-raw-device enp0s3
iface enp0s3.90 inet static
 address 192.168.90.1
 subnet 255.255.255.0
 newtwork 192.168.90.0
 vlan-raw-device enp0s3

Then restart networking (sudo service networking restart).

If all is good, you should get something a little like this:

stuart@stuart-VirtualBox:~$ ifconfig
enp0s3    Link encap:Ethernet  HWaddr 08:00:27:08:ad:15 
          inet addr:192.168.1.230  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::e64d:1dad:f4bf:13af/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:14001 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3546 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:13838449 (13.8 MB)  TX bytes:279173 (279.1 KB)

enp0s3.5  Link encap:Ethernet  HWaddr 08:00:27:08:ad:15 
          inet addr:192.168.5.1  Bcast:192.168.5.255  Mask:255.255.255.0
          inet6 addr: fe80::a00:27ff:fe08:ad15/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:41 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:5237 (5.2 KB)

enp0s3.90 Link encap:Ethernet  HWaddr 08:00:27:08:ad:15 
          inet addr:192.168.90.1  Bcast:192.168.90.255  Mask:255.255.255.0
          inet6 addr: fe80::a00:27ff:fe08:ad15/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:41 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:5268 (5.2 KB)

lo        Link encap:Local Loopback 
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:309 errors:0 dropped:0 overruns:0 frame:0
          TX packets:309 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1 
          RX bytes:23482 (23.4 KB)  TX bytes:23482 (23.4 KB)

stuart@stuart-VirtualBox:~$

Creating the virtual hosts in Apache and generating certificates

Creating the virtual hosts was a bit more tricky, there was a lot of trial and error!

I needed the certificates to be created easily, thankfully this can all be done in one line:

sudo openssl req -x509 -newkey rsa:2048 -nodes -keyout /etc/ssl/private/server.key -out /etc/ssl/certs/server.pem -days 365 -subj "/C=GB/ST=London/L=London/O=802101/OU=Org/CN=server.802101.com"

Now we just need to do this a number of times!

#!/bin/bash
while IFS=":" read v1 v2 v3 v4 v5 v6 _
do
  echo "Site name is" $v1
  echo "Creating SSL cert for" $v1
  sudo openssl req -x509 -newkey rsa:2048 -nodes -keyout /etc/ssl/private/$v1.key -out /etc/ssl/certs/$v1.pem -days 365 -subj "/C=GB/ST=London/L=London/O=802101/OU=Org/CN=$v1"
  echo "Passing" $v1 "to virtualhost.sh"
  sudo ./virtualhost.sh create $v1 $v4 $v3 
  echo "External IP is" $v2
  echo "Internal IP is" $v3
  echo "Creating an interface with IP address" $v3
  sudo ip addr add $v3/$v5 dev enp0s3.$v6
  echo "Protocol is" $v4;
  echo " "
done < testsites.txt
sudo /etc/init.d/apache2 reload

This script uses IFS (Internal Field Separator) to read values from an input file (testsites.txt). Each field is separated by a colon (:) and assigned a variable, v1 – v6.

It returns the site name and creates an SSL cert for it. We then pass the site name, domain IP address and port through to the script “virtualhost.sh” to creat the virtual hosts. It returns some variables and creates an additional IP address with the IP address and subnet mask on the enp0s3 VLAN interface.

Finally, it restarts Apache.

Be careful with how many spaces there are in the script above. Too many spaces at the start of each line resulted in this:

stuart@stuart-VirtualBox:~$ ./testrun.sh
./testrun.sh: line 4:  : command not found
./testrun.sh: line 4:  : command not found
./testrun.sh: line 4:  : command not found
./testrun.sh: line 4:  : command not found
./testrun.sh: line 4:  : command not found

Two spaces is good for indentation, too many and an error occurs.

I have called this script “catsites.sh”. Remember to “sudo chmod a+x catsites.sh” and the virtualhost.sh file as well.

The virtualhost.sh script is not mine. It came from here: https://github.com/RoverWire/virtualhost/blob/master/virtualhost.sh.

I have modified it though to suit my purposes.

So, I changed:

### Set default parameters
action=$1
domain=$2
rootDir=$3
owner=$(who am i | awk '{print $1}')

To:

### Set default parameters
action=$1
domain=$2
domainPort=$3
domainIP=$4
rootDir=$5
owner=$(who am i | awk '{print $1}')

I then changed the echo statement in:

### check if directory exists or not
 if ! [ -d $rootDir ]; then
     ### create the directory
     mkdir $rootDir
     ### give permission to root dir
    chmod 755 $rootDir
    ### write test file in the new domain dir
    if ! echo "<?php echo phpinfo(); ?>" > $rootDir/phpinfo.php
    then
      echo $"ERROR: Not able to write in file $rootDir/phpinfo.php. Please check permissions"
      exit;
    else
      echo $"Added content to $rootDir/phpinfo.php"
    fi
 fi

to:

if ! echo "<html><head></head><body><br><br><br>$domain<br></body></html>" > $rootDir/index.html

Next, I changed to the virtual host part from:

<VirtualHost *:80>
   ServerAdmin $email
   ServerName $domain
   ServerAlias $domain
   DocumentRoot $rootDir
   <Directory />
      AllowOverride All
   </Directory>
   <Directory $rootDir>
      Options Indexes FollowSymLinks MultiViews
      AllowOverride all
      Require all granted
   </Directory>
   ErrorLog /var/log/apache2/$domain-error.log
   LogLevel error
   CustomLog /var/log/apache2/$domain-access.log combined
 </VirtualHost>" > $sitesAvailabledomain

To:

 Listen $domain:$domainPort
 <VirtualHost $domain:80>
    ServerAdmin $email
    ServerName $domain
    ServerAlias $domain
    DocumentRoot $rootDir
    <Directory />
       AllowOverride All
    </Directory>
    <Directory $rootDir>
       Options Indexes FollowSymLinks MultiViews
       AllowOverride all
       Require all granted
       DirectoryIndex index.html
    </Directory>
    ErrorLog /var/log/apache2/$domain-error.log
    LogLevel error
    CustomLog /var/log/apache2/$domain-access.log combined
 </VirtualHost>
 <VirtualHost $domain:$domainPort>
    ServerAdmin $email
    ServerName $domain
    ServerAlias $domain
    DocumentRoot $rootDir
    ErrorLog /var/log/apache2/$domain-error.log
    LogLevel error
    CustomLog /var/log/apache2/$domain-access.log combined
    SSLEngine on
    SSLOptions +StrictRequire
    SSLCertificateFile /etc/ssl/certs/$domain.pem
    SSLCertificateKeyFile /etc/ssl/private/$domain.key
 </VirtualHost>" > $sitesAvailabledomain

Because we are using the domain names, we also need to change:

### Add domain in /etc/hosts
 if ! echo "127.0.0.1 $domain" >> /etc/hosts

To:

 ### Add domain in /etc/hosts
 if ! echo "$domainIP $domain" >> /etc/hosts

Lastly, comment out the line that restarts Apache:

 ### enable website
 a2ensite $domain
 ### restart Apache
 # /etc/init.d/apache2 reload

Because Apache may take a little time to start, we don’t want it to be stopping and starting all the time, so this goes at the end of the master script instead.

The final part is to create the input file, which looks like this:

server1.802101.com:192.0.2.1:192.168.5.10:443:24:5
server2.802101.com:192.0.2.2:192.168.5.11:443:24:5
server3.802101.com:192.0.2.3:192.168.5.12:443:24:5
server4.802101.com:192.0.2.4:192.168.5.13:443:24:5
server5.802101.com:192.0.2.5:192.168.90.10:443:24:90
server6.802101.com:192.0.2.6:192.168.90.11:443:24:90
server7.802101.com:192.0.2.7:192.168.90.12:443:24:90
server8.802101.com:192.0.2.8:192.168.90.13:443:24:90
server9.802101.com:192.0.2.9:192.168.90.14:443:24:90
server10.802101.com:192.0.2.10:192.168.90.15:1443:24:90
server11.802101.com:192.0.2.11:192.168.90.16:2443:24:90
server12.802101.com:192.0.2.12:192.168.90.17:3443:24:90
server13.802101.com:192.0.2.13:192.168.90.18:4443:24:90

We have thirteen websites, each has an external IP address (useful if you want one file to use on another server to do external testing on, the internal IP address, the port, the subnet mask and the VLAN.

Running the script

Ready? Let’s give it a go:

Call the script using “sudo ./catsites.sh”.

I won’t put all of the output here, but this is what the last one should look like:

Site name is server13.802101.com
Creating SSL cert for server13.802101.com
Generating a 2048 bit RSA private key
.................................................................+++

................................................................................+++

writing new private key to '/etc/ssl/private/server13.802101.com.key'
-----
Passing server13.802101.com to virtualhost.sh
Added content to /var/www/server13802101com/phpinfo.php
New Virtual Host Created
Host added to /etc/hosts file 
Enabling site server13.802101.com.
To activate the new configuration, you need to run:
  service apache2 reload
Complete! 
You now have a new Virtual Host 
Your new host is: http://server13.802101.com 
And its located at /var/www/server13802101com
External IP is 192.0.2.13
Internal IP is 192.168.90.18
Creating an interface with IP address 192.168.90.18
Protocol is 4443
[ ok ] Reloading apache2 configuration (via systemctl): apache2.service.
stuart@stuart-VirtualBox:~$

Checking it works

As we are using the FQDNs of our virtual hosts, we should check our /etc/hosts file:

stuart@stuart-VirtualBox:~$ cat /etc/hosts
127.0.0.1 localhost
127.0.1.1 stuart-VirtualBox
# The following lines are desirable for IPv6 capable hosts
::1     ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
192.168.5.10 server1.802101.com
192.168.5.11 server2.802101.com
192.168.5.12 server3.802101.com
192.168.5.13 server4.802101.com
192.168.90.10 server5.802101.com
192.168.90.11 server6.802101.com
192.168.90.12 server7.802101.com
192.168.90.13 server8.802101.com
192.168.90.14 server9.802101.com
192.168.90.15 server10.802101.com
192.168.90.16 server11.802101.com
192.168.90.17 server12.802101.com
192.168.90.18 server13.802101.com

stuart@stuart-VirtualBox:~$

Next, that Apache is running with no errors:

stuart@stuart-VirtualBox:~$ sudo service apache2 status
● apache2.service - LSB: Apache2 web server
   Loaded: loaded (/etc/init.d/apache2; bad; vendor preset: enabled)
  Drop-In: /lib/systemd/system/apache2.service.d
           └─apache2-systemd.conf
   Active: active (running) since Fri 2018-05-04 10:43:45 BST; 1h 14min ago
     Docs: man:systemd-sysv-generator(8)
  Process: 11471 ExecStop=/etc/init.d/apache2 stop (code=exited, status=0/SUCCESS)
  Process: 14828 ExecReload=/etc/init.d/apache2 reload (code=exited, status=0/SUCCESS)
  Process: 11494 ExecStart=/etc/init.d/apache2 start (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/apache2.service
           ├─11512 /usr/sbin/apache2 -k start
           ├─14863 /usr/sbin/apache2 -k start
           ├─14864 /usr/sbin/apache2 -k start
           ├─14865 /usr/sbin/apache2 -k start
           ├─14866 /usr/sbin/apache2 -k start
           └─14867 /usr/sbin/apache2 -k start
May 04 10:56:11 stuart-VirtualBox apache2[13762]:  *
May 04 10:56:11 stuart-VirtualBox systemd[1]: Reloaded LSB: Apache2 web server.
May 04 11:52:33 stuart-VirtualBox systemd[1]: Reloading LSB: Apache2 web server.
May 04 11:52:33 stuart-VirtualBox apache2[14403]:  * Reloading Apache httpd web server apache2
May 04 11:52:33 stuart-VirtualBox apache2[14403]:  *
May 04 11:52:33 stuart-VirtualBox systemd[1]: Reloaded LSB: Apache2 web server.
May 04 11:53:06 stuart-VirtualBox systemd[1]: Reloading LSB: Apache2 web server.
May 04 11:53:06 stuart-VirtualBox apache2[14828]:  * Reloading Apache httpd web server apache2
May 04 11:53:06 stuart-VirtualBox apache2[14828]:  *
May 04 11:53:06 stuart-VirtualBox systemd[1]: Reloaded LSB: Apache2 web server.
stuart@stuart-VirtualBox:~$