Overview
In our last post, we configured a Ceph storage cluster, which we’ll be using as the storage for our virtual machines that we’ll be using to host Kubernetes.
Before we get to that, however, we need to configure our DNS environment. Recall that, in part 2, we configured a Windows server to act as our DNS and DHCP server. For this, you don’t need a Windows Server installation, you just need an RFC 2136-compliant DNS server like PowerDNS. Keep in mind that RFC 2136 has some security vulnerabilities, so don’t use dynamic DNS in a high-security environment or an environment wherein untrusted devices may be on the network.
To provision our DNS names, we’ll be interacting with our DNS server via the Terraform DNS provider. Terraform’s easy to set up, so we won’t be covering that.
Note that this solution requires a fair amount of work in Bash. Other solutions like Chef/Puppet/Ansible/Salt may strictly be better-suited for this sort of work, but since our environment is quite homogeneous (99% Debian), we’ll just script this using basic tools.
A note on Terraform Typing
Terraform’s variable/parameter management system, specifically its inability to share variables between modules can result in some duplication between modules. This isn’t the cleanest, but I don’t feel like it’s a big enough problem to warrant bringing in other tools like Terragrunt. Terraform’s type-system supports a flavor of structural typing wherein you can declare a variable with a subset of another variable’s fields and supply the first variable as the second variable. For instance:
variable "A" { | |
type = object({ | |
name = string | |
ip = string | |
}) | |
} | |
variable "B" { | |
type = object({ | |
name = string | |
}) | |
} |
Usage:
// module B declares B, and can use a value of type A as the value for b:B b = var.a
We’ll be using this feature to share variable values across modules.
Setup
Since we’re reserving the upper range of our /24 subnet for MetalLB dynamically-provisioned IP addresses, we’ll configure the worker nodes with static IP addresses assigned from the lower range of our subnet. Since we have a small number of physical machines, we’ll opt for fewer, larger worker-nodes instead of more, smaller worker-nodes. The subnet-ranges we’ll select for these roles are as follows:
2-10
: infrastructure (domain-controller, physical node IPs, etcd VMs, Kubernetes leader nodes)10-30
: static addresses (worker nodes)30-200
: DHCP leases for dynamic devices on the network + MetalLB IPs200-255
: available
To set that up in Windows Server:
- Navigate to the
Server Manager
- From the top-right
Tools
menu, selectDHCP
- Right-click on the IPv4 node under your domain-controller host-name and select
New Scope
- Provide a
Start address
of<subnet prefix>
.30
and an end-address of200
- Continue through the wizard providing your gateway addresses and your exclusions. For the exclusions, select
<subnet prefix>
-1 through 30. - Finish the wizard and click
Create
to create your scope. - Right-click on the created scope node in the tree. Select the
DNS
tab, then enable DNS dynamic updates. Select theDynamically update DNS records only if requested by the DHCP clients
. - Click
Ok
to save your changes.
Configure your DD-WRT Router
Navigate back to your gateway router. Most capable routers will have an option to either act as a DHCP server (the default), or to act as a DHCP forwarder. For DD-WRT, select the `DHCP Forwarder option and provide the IP address of your DHCP server as the target.
Configure your DNS Server
Go back to your domain controller.
- From the
Server Manager
, selectDNS
- Locate the
Forward Lookup Zone
node in theDNS Manager
tree that you created (ours wassunshower.io.
Right-click on theForward Lookup Zone
node and clickProperties
- In the
General
Tab, select theNonsecure and Secure
option from theDynamic updates
configuration. Note: This is not suitable for an open-network or one admitting untrusted/unknown devices - Click
Ok
to apply.
Congrats! We’re now ready to provision our DNS in Terraform.
Terraform Configuration
Note: We’ve provided our Terraform configurations here, so feel free to point them at your infrastructure if you’ve been following along instead of creating your own.
In the directory of your Terraform project ($TFDIR), create a directory called dns
.
1. In your dns
directory, create 3 files: main.tf
, variables.tf
, and outputs.tf
Main.tf
Your main.tf
file should contain:
terraform { | |
required_providers { | |
dns = { | |
source = "hashicorp/dns" | |
} | |
} | |
} | |
/** | |
provision DNS entries for each host in the `hosts` list | |
*/ | |
resource "dns_a_record_set" "virtual_machine_dns" { | |
for_each = {for vm in var.hosts: vm.name => vm} | |
zone = var.dns_server.zone | |
name = each.value.name | |
addresses = [ | |
each.value.ip | |
] | |
} | |
/** | |
provision the DNS A-record for the API server | |
*/ | |
resource "dns_a_record_set" "api_server_dns" { | |
addresses = [ | |
var.api_ip | |
] | |
zone = var.dns_server.zone | |
name = var.api_dns | |
} |
Now, in your Terraform project $TFDIR
(the parent of the dns
directory you’re currently in, create main.tf
, outputs.tf
, and variables.tf
. Symlink the dns/variables.tf
to this directory as dns-variables.tf
via ln -s $(pwd)/vm-variables.tf $(pwd)/dns/variables.tf
to make your DNS variables available to your root configuration.
Parent module
In your $TFDIR/main.tf
, add the following block:
terraform { | |
required_version = ">=0.14" | |
required_providers { | |
dns = { | |
source = "hashicorp/dns" | |
version = "3.0.1" | |
} | |
} | |
} |
Configure the DNS provider to point to your RFC-2136-compliant DNS server:
provider "dns" { | |
update { | |
server = var.dns_server.server | |
} | |
} |
Call the dns
submodule:
/** | |
Create DNS entries for virtual machines | |
*/ | |
module "dns_configuration" { | |
for_each = var.cluster_nodes | |
source = "./dns" | |
dns_server = var.dns_server | |
hosts = each.value | |
api_dns = var.api_server | |
api_domain = var.domain | |
api_ip = var.load_balancer | |
} |
At this point, we should be ready to actually provide our Terraform variables.
- Create a file in your
$TFDIR
directory calledconfiguration.auto.tfvars
–Terraform will automatically pick up any in the root directory according to the naming convention*.auto.tfvars
. Be sure to never check these into source-control. I add the .gitignore rule**/*.tfvars
In your $TFDIR/configuration.auto.tfvars
file, add the following configurations (substitute with your values):
dns_server = { | |
// replace with your zone-name (configured above) | |
zone = "sunshower.cloud." // note the trailing period here--it's mandatory | |
// replace with your DNS server's IP | |
server = "192.168.1.2" | |
} | |
// this generates the DNS configuration for `kubernetes.sunshower.cloud` | |
api_dns = "kubernetes" | |
api_domain = "sunshower.cloud" | |
api_ip = "192.168.1.10" // provide the API you |
Finally, we’ll add our Kubernetes cluster DNS configurations:
cluster_nodes = { | |
etcd_nodes = [ | |
{ | |
name = "etcd-1" | |
ip = "192.168.1.5" | |
}, | |
{ | |
name = "etcd-2" | |
ip = "192.168.1.6" | |
}, | |
{ | |
name = "etcd-3" | |
ip = "192.168.1.7" | |
} | |
], | |
k8s_leaders = [ | |
{ | |
name = "k8s-leader-1" | |
ip = "192.168.1.8" | |
}, | |
{ | |
name = "k8s-leader-2" | |
ip = "192.168.1.9" | |
} | |
], | |
k8s_workers = [ | |
{ | |
name = "k8s-worker-1" | |
ip = "192.168.1.11" | |
}, | |
{ | |
name = "k8s-worker-2" | |
ip = "192.168.1.12" | |
}, | |
{ | |
name = "k8s-worker-3" | |
ip = "192.168.1.13" | |
} | |
] | |
} |
At this point, running terraform apply --auto-approve
should quickly generate the DNS entries. You should navigate to your DNS server and refresh the forward lookup zone to confirm that your entries exist.
Conclusion
In this part, we configured DHCP, DNS, and provided a Terrform module that provisions DNS entries for each of the VMs that we’ll provision in subsequent posts. Next time, we’ll create a Terraform module that will provision virtual machines of the correct roles for our Kubernetes cluster. Our final post of the series will show how to actually deploy the Kubernetes cluster–stay tuned!