A comprehensive Vagrant-based development environment for OKD (OpenShift Origin) 3.11, providing a multi-node cluster setup with authentication, persistent volumes, service registry, and S2I examples.

Overview
Architecture
Prerequisites
Installation
Configuration Options
Post-Installation Setup
Examples and Use Cases
Troubleshooting
Advanced Configuration

Overview

This project creates a complete OKD 3.11 cluster using Vagrant with the following features:

Multi-node setup: Master, worker nodes, and dedicated storage/services node
Automated provisioning: Full cluster deployment with Ansible playbooks
Multiple deployment options: Full cluster, all-in-one, or custom configurations
Comprehensive examples: Authentication, persistent volumes, S2I builds, and more
Production-ready: HAProxy load balancer configuration included

Architecture

Full Cluster Setup (Default)

Machine	Address	Memory	CPUs	Roles
okd.example.com	172.27.11.10	8GB	4	master, infra, etcd
node1.example.com	172.27.11.20	4GB	2	compute node
node2.example.com	172.27.11.30	4GB	2	compute node
extras.example.com	172.27.11.40	256MB	1	storage (NFS), LDAP

Total Resources Required: ~16.25GB RAM, 9 CPU cores

Network Configuration

Private Network: 172.27.11.0/24
Public Access: Through nip.io wildcard DNS (*.172-27-11-10.nip.io)
Load Balancer: HAProxy configuration provided for production use

Prerequisites

Software Requirements

Vagrant (latest version)
VirtualBox or libvirt (KVM)
Minimum 16GB RAM available for full setup
~50GB disk space for all VMs

Supported Platforms

Linux (recommended)
macOS
Windows (with some limitations)

Installation

Quick Start - Full Cluster

git clone <repository-url>
cd okdv3
vagrant up

The installation process will:

Create and configure 4 virtual machines
Install required packages and dependencies
Run Ansible playbooks for OKD installation:
- /root/openshift-ansible/playbooks/prerequisites.yml
- /root/openshift-ansible/playbooks/deploy_cluster.yml

⏱️ Installation Time: 60-90 minutes depending on hardware

Low Memory Setup (8GB or less)

For systems with limited memory, use the all-in-one configuration:

git clone <repository-url>
cd okdv3
mv Vagrantfile Vagrantfile.full
mv Vagrantfile.allinone Vagrantfile
vagrant up

This creates only 2 VMs:

Master: 4GB RAM (all services)
Extras: 256MB RAM (NFS + LDAP)

Configuration Options

Vagrant Provider Support

The project supports both VirtualBox and libvirt providers:

# VirtualBox (default)
vagrant up

# libvirt/KVM
vagrant up --provider=libvirt

Memory Optimization

To reduce memory usage, edit the Ansible inventory (files/hosts) and disable metrics:

openshift_metrics_install_metrics=false

This reduces master memory requirements from 8GB to ~2GB.

Disabled Services (for performance)

The following services are disabled by default:

openshift_logging_install_logging=false
openshift_enable_olm=false
openshift_enable_service_catalog=false
openshift_cluster_monitoring_operator_install=false

Post-Installation Setup

1. Access the Web Console

Add the hostname to your system's hosts file:

Linux/macOS:

echo '172.27.11.10 okd.example.com' | sudo tee -a /etc/hosts

Windows: Edit C:\Windows\System32\drivers\etc\hosts and add:

172.27.11.10 okd.example.com

Web Console: https://okd.example.com:8443
Username: developer
Password: 4linux

3. Accept SSL Certificates

Visit and accept the self-signed certificate for metrics:

https://hawkular-metrics.172-27-11-10.nip.io

4. CLI Access

SSH into the master node:

vagrant ssh master
oc login -u developer -p 4linux

Examples and Use Cases

1. Authentication (examples/authentication/)

Configure different authentication methods:

HTPasswd: File-based authentication
LDAP: Directory-based authentication (pre-configured)

Example HTPasswd setup:

vagrant ssh master
sudo htpasswd -bc /etc/origin/master/htpasswd myuser mypassword
# Update master-config.yaml and restart services

2. Persistent Volumes (examples/persistent-volumes/)

Pre-configured NFS storage on the extras node:

# Create NFS exports on extras node
vagrant ssh extras
sudo mkdir -p /srv/nfs/v{0,1,2,3,4}
sudo chmod 0700 /srv/nfs/v{0,1,2,3,4}
sudo chown nfsnobody: /srv/nfs/v{0,1,2,3,4}

# Enable SELinux for NFS on all nodes
sudo setsebool -P virt_use_nfs 1

Deploy persistent volumes:

vagrant ssh master
oc create -f /vagrant/examples/persistent-volumes/nfs-pv.yml
oc create -f /vagrant/examples/persistent-volumes/cache-pvc.yml

3. Container Registry (examples/registry/)

Set up and configure the integrated container registry for storing images.

4. Source-to-Image (S2I) (examples/s2i/)

Custom S2I builder for lighttpd web server:

Custom Dockerfile
Build and run scripts
Example application template

Deploy the S2I example:

vagrant ssh master
oc create -f /vagrant/examples/template/lighttpd.yml
oc new-app lighttpd-s2i

Troubleshooting

Common Issues

1. Insufficient Memory

Error: VM fails to start or OKD pods crash Solution:

Use all-in-one configuration
Disable metrics: Set openshift_metrics_install_metrics=false

2. Network Issues

Error: Cannot access web console Solution:

Verify hosts file entry
Check VM network: vagrant ssh master -c "ip addr show"
Ensure no firewall blocking ports 8443, 80, 443

3. Certificate Issues

Error: SSL certificate warnings Solution:

Accept self-signed certificates in browser
For CLI: oc login --insecure-skip-tls-verify=true

4. Storage Issues

Error: PVCs stuck in pending Solution:

# Check NFS service on extras node
vagrant ssh extras
sudo systemctl status nfs-server

# Verify exports
sudo exportfs -v

Debugging Commands

# Check cluster status
vagrant ssh master
oc get nodes
oc get pods --all-namespaces

# Check services
sudo systemctl status origin-master-api
sudo systemctl status origin-master-controllers

# View logs
sudo journalctl -u origin-master-api -f
sudo journalctl -u origin-master-controllers -f

Advanced Configuration

Custom Ansible Inventory

The main configuration is in files/hosts. Key sections:

[OSEv3:vars]
# Authentication
openshift_master_identity_providers=[{'name': 'HTPASSWD', 'challenge': 'true', 'login': 'true', 'kind':'HTPasswdPasswordIdentityProvider', 'mappingMethod': 'claim'}]

# Networking
openshift_master_default_subdomain='172-27-11-10.nip.io'

# Docker configuration
openshift_docker_options='--selinux-enabled --log-driver=journald --signature-verification=false --insecure-registry=172.30.0.0/16 --exec-opt native.cgroupdriver=systemd'

# Disable checks (for development)
openshift_disable_check='disk_availability,memory_availability,docker_storage,package_availability,docker_image_availability,package_version'

HAProxy Configuration

Production-ready HAProxy configuration is provided in haproxy/ directory:

Load balancer configuration
SSL termination
Backend mapping for OpenShift routes

Provisioning Scripts

Custom provisioning scripts in provision/ directory:

master.sh: Master node setup
node.sh: Worker node configuration
extras.sh: Storage and LDAP setup
allinone.sh: All-in-one deployment

File Structure

okdv3/
├── Vagrantfile              # Main cluster configuration
├── Vagrantfile.allinone     # Single-node configuration
├── Vagrantfile.full         # Full cluster backup
├── files/
│   ├── hosts               # Ansible inventory
│   ├── hosts-allinone      # Single-node inventory
│   ├── key                 # SSH private key
│   ├── key.pub             # SSH public key
│   └── *.ldif              # LDAP configuration
├── provision/              # Provisioning scripts
├── examples/               # Usage examples
│   ├── authentication/    # Auth configuration
│   ├── persistent-volumes/ # Storage examples
│   ├── registry/          # Container registry
│   ├── s2i/               # Source-to-Image
│   └── template/          # Application templates
└── haproxy/               # Load balancer config

Contributing

Feel free to submit issues, feature requests, and pull requests to improve this OKD development environment.

License

This project is provided as-is for educational and development purposes.

README.md

OKD 3.11 Vagrant Development Environment

Table of Contents