339 lines
9.1 KiB
Markdown
339 lines
9.1 KiB
Markdown
# OKD 3.11 Vagrant Development Environment
|
|
|
|
A comprehensive Vagrant-based development environment for OKD (OpenShift Origin) 3.11, providing a multi-node cluster setup with authentication, persistent volumes, service registry, and S2I examples.
|
|
|
|
## Table of Contents
|
|
|
|
- [Overview](#overview)
|
|
- [Architecture](#architecture)
|
|
- [Prerequisites](#prerequisites)
|
|
- [Installation](#installation)
|
|
- [Configuration Options](#configuration-options)
|
|
- [Post-Installation Setup](#post-installation-setup)
|
|
- [Examples and Use Cases](#examples-and-use-cases)
|
|
- [Troubleshooting](#troubleshooting)
|
|
- [Advanced Configuration](#advanced-configuration)
|
|
|
|
## Overview
|
|
|
|
This project creates a complete OKD 3.11 cluster using Vagrant with the following features:
|
|
|
|
- **Multi-node setup**: Master, worker nodes, and dedicated storage/services node
|
|
- **Automated provisioning**: Full cluster deployment with Ansible playbooks
|
|
- **Multiple deployment options**: Full cluster, all-in-one, or custom configurations
|
|
- **Comprehensive examples**: Authentication, persistent volumes, S2I builds, and more
|
|
- **Production-ready**: HAProxy load balancer configuration included
|
|
|
|
## Architecture
|
|
|
|
### Full Cluster Setup (Default)
|
|
|
|
| Machine | Address | Memory | CPUs | Roles |
|
|
|---------------------|---------------|--------|------|--------------------------|
|
|
| okd.example.com | 172.27.11.10 | 8GB | 4 | master, infra, etcd |
|
|
| node1.example.com | 172.27.11.20 | 4GB | 2 | compute node |
|
|
| node2.example.com | 172.27.11.30 | 4GB | 2 | compute node |
|
|
| extras.example.com | 172.27.11.40 | 256MB | 1 | storage (NFS), LDAP |
|
|
|
|
**Total Resources Required**: ~16.25GB RAM, 9 CPU cores
|
|
|
|
### Network Configuration
|
|
|
|
- **Private Network**: 172.27.11.0/24
|
|
- **Public Access**: Through nip.io wildcard DNS (*.172-27-11-10.nip.io)
|
|
- **Load Balancer**: HAProxy configuration provided for production use
|
|
|
|
## Prerequisites
|
|
|
|
### Software Requirements
|
|
|
|
- **Vagrant** (latest version)
|
|
- **VirtualBox** or **libvirt** (KVM)
|
|
- **Minimum 16GB RAM** available for full setup
|
|
- **~50GB disk space** for all VMs
|
|
|
|
### Supported Platforms
|
|
|
|
- Linux (recommended)
|
|
- macOS
|
|
- Windows (with some limitations)
|
|
|
|
## Installation
|
|
|
|
### Quick Start - Full Cluster
|
|
|
|
```bash
|
|
git clone <repository-url>
|
|
cd okdv3
|
|
vagrant up
|
|
```
|
|
|
|
The installation process will:
|
|
1. Create and configure 4 virtual machines
|
|
2. Install required packages and dependencies
|
|
3. Run Ansible playbooks for OKD installation:
|
|
- `/root/openshift-ansible/playbooks/prerequisites.yml`
|
|
- `/root/openshift-ansible/playbooks/deploy_cluster.yml`
|
|
|
|
**⏱️ Installation Time**: 60-90 minutes depending on hardware
|
|
|
|
### Low Memory Setup (8GB or less)
|
|
|
|
For systems with limited memory, use the all-in-one configuration:
|
|
|
|
```bash
|
|
git clone <repository-url>
|
|
cd okdv3
|
|
mv Vagrantfile Vagrantfile.full
|
|
mv Vagrantfile.allinone Vagrantfile
|
|
vagrant up
|
|
```
|
|
|
|
This creates only 2 VMs:
|
|
- **Master**: 4GB RAM (all services)
|
|
- **Extras**: 256MB RAM (NFS + LDAP)
|
|
|
|
## Configuration Options
|
|
|
|
### Vagrant Provider Support
|
|
|
|
The project supports both VirtualBox and libvirt providers:
|
|
|
|
```ruby
|
|
# VirtualBox (default)
|
|
vagrant up
|
|
|
|
# libvirt/KVM
|
|
vagrant up --provider=libvirt
|
|
```
|
|
|
|
### Memory Optimization
|
|
|
|
To reduce memory usage, edit the Ansible inventory (`files/hosts`) and disable metrics:
|
|
|
|
```ini
|
|
openshift_metrics_install_metrics=false
|
|
```
|
|
|
|
This reduces master memory requirements from 8GB to ~2GB.
|
|
|
|
### Disabled Services (for performance)
|
|
|
|
The following services are disabled by default:
|
|
- `openshift_logging_install_logging=false`
|
|
- `openshift_enable_olm=false`
|
|
- `openshift_enable_service_catalog=false`
|
|
- `openshift_cluster_monitoring_operator_install=false`
|
|
|
|
## Post-Installation Setup
|
|
|
|
### 1. Access the Web Console
|
|
|
|
Add the hostname to your system's hosts file:
|
|
|
|
**Linux/macOS**:
|
|
```bash
|
|
echo '172.27.11.10 okd.example.com' | sudo tee -a /etc/hosts
|
|
```
|
|
|
|
**Windows**:
|
|
Edit `C:\Windows\System32\drivers\etc\hosts` and add:
|
|
```
|
|
172.27.11.10 okd.example.com
|
|
```
|
|
|
|
### 2. Login Credentials
|
|
|
|
- **Web Console**: https://okd.example.com:8443
|
|
- **Username**: `developer`
|
|
- **Password**: `4linux`
|
|
|
|
### 3. Accept SSL Certificates
|
|
|
|
Visit and accept the self-signed certificate for metrics:
|
|
- https://hawkular-metrics.172-27-11-10.nip.io
|
|
|
|
### 4. CLI Access
|
|
|
|
SSH into the master node:
|
|
```bash
|
|
vagrant ssh master
|
|
oc login -u developer -p 4linux
|
|
```
|
|
|
|
## Examples and Use Cases
|
|
|
|
### 1. Authentication (examples/authentication/)
|
|
|
|
Configure different authentication methods:
|
|
- **HTPasswd**: File-based authentication
|
|
- **LDAP**: Directory-based authentication (pre-configured)
|
|
|
|
Example HTPasswd setup:
|
|
```bash
|
|
vagrant ssh master
|
|
sudo htpasswd -bc /etc/origin/master/htpasswd myuser mypassword
|
|
# Update master-config.yaml and restart services
|
|
```
|
|
|
|
### 2. Persistent Volumes (examples/persistent-volumes/)
|
|
|
|
Pre-configured NFS storage on the extras node:
|
|
|
|
```bash
|
|
# Create NFS exports on extras node
|
|
vagrant ssh extras
|
|
sudo mkdir -p /srv/nfs/v{0,1,2,3,4}
|
|
sudo chmod 0700 /srv/nfs/v{0,1,2,3,4}
|
|
sudo chown nfsnobody: /srv/nfs/v{0,1,2,3,4}
|
|
|
|
# Enable SELinux for NFS on all nodes
|
|
sudo setsebool -P virt_use_nfs 1
|
|
```
|
|
|
|
Deploy persistent volumes:
|
|
```bash
|
|
vagrant ssh master
|
|
oc create -f /vagrant/examples/persistent-volumes/nfs-pv.yml
|
|
oc create -f /vagrant/examples/persistent-volumes/cache-pvc.yml
|
|
```
|
|
|
|
### 3. Container Registry (examples/registry/)
|
|
|
|
Set up and configure the integrated container registry for storing images.
|
|
|
|
### 4. Source-to-Image (S2I) (examples/s2i/)
|
|
|
|
Custom S2I builder for lighttpd web server:
|
|
- Custom Dockerfile
|
|
- Build and run scripts
|
|
- Example application template
|
|
|
|
Deploy the S2I example:
|
|
```bash
|
|
vagrant ssh master
|
|
oc create -f /vagrant/examples/template/lighttpd.yml
|
|
oc new-app lighttpd-s2i
|
|
```
|
|
|
|
## Troubleshooting
|
|
|
|
### Common Issues
|
|
|
|
#### 1. Insufficient Memory
|
|
**Error**: VM fails to start or OKD pods crash
|
|
**Solution**:
|
|
- Use all-in-one configuration
|
|
- Disable metrics: Set `openshift_metrics_install_metrics=false`
|
|
|
|
#### 2. Network Issues
|
|
**Error**: Cannot access web console
|
|
**Solution**:
|
|
- Verify hosts file entry
|
|
- Check VM network: `vagrant ssh master -c "ip addr show"`
|
|
- Ensure no firewall blocking ports 8443, 80, 443
|
|
|
|
#### 3. Certificate Issues
|
|
**Error**: SSL certificate warnings
|
|
**Solution**:
|
|
- Accept self-signed certificates in browser
|
|
- For CLI: `oc login --insecure-skip-tls-verify=true`
|
|
|
|
#### 4. Storage Issues
|
|
**Error**: PVCs stuck in pending
|
|
**Solution**:
|
|
```bash
|
|
# Check NFS service on extras node
|
|
vagrant ssh extras
|
|
sudo systemctl status nfs-server
|
|
|
|
# Verify exports
|
|
sudo exportfs -v
|
|
```
|
|
|
|
### Debugging Commands
|
|
|
|
```bash
|
|
# Check cluster status
|
|
vagrant ssh master
|
|
oc get nodes
|
|
oc get pods --all-namespaces
|
|
|
|
# Check services
|
|
sudo systemctl status origin-master-api
|
|
sudo systemctl status origin-master-controllers
|
|
|
|
# View logs
|
|
sudo journalctl -u origin-master-api -f
|
|
sudo journalctl -u origin-master-controllers -f
|
|
```
|
|
|
|
## Advanced Configuration
|
|
|
|
### Custom Ansible Inventory
|
|
|
|
The main configuration is in `files/hosts`. Key sections:
|
|
|
|
```ini
|
|
[OSEv3:vars]
|
|
# Authentication
|
|
openshift_master_identity_providers=[{'name': 'HTPASSWD', 'challenge': 'true', 'login': 'true', 'kind':'HTPasswdPasswordIdentityProvider', 'mappingMethod': 'claim'}]
|
|
|
|
# Networking
|
|
openshift_master_default_subdomain='172-27-11-10.nip.io'
|
|
|
|
# Docker configuration
|
|
openshift_docker_options='--selinux-enabled --log-driver=journald --signature-verification=false --insecure-registry=172.30.0.0/16 --exec-opt native.cgroupdriver=systemd'
|
|
|
|
# Disable checks (for development)
|
|
openshift_disable_check='disk_availability,memory_availability,docker_storage,package_availability,docker_image_availability,package_version'
|
|
```
|
|
|
|
### HAProxy Configuration
|
|
|
|
Production-ready HAProxy configuration is provided in `haproxy/` directory:
|
|
- Load balancer configuration
|
|
- SSL termination
|
|
- Backend mapping for OpenShift routes
|
|
|
|
### Provisioning Scripts
|
|
|
|
Custom provisioning scripts in `provision/` directory:
|
|
- `master.sh`: Master node setup
|
|
- `node.sh`: Worker node configuration
|
|
- `extras.sh`: Storage and LDAP setup
|
|
- `allinone.sh`: All-in-one deployment
|
|
|
|
### File Structure
|
|
|
|
```
|
|
okdv3/
|
|
├── Vagrantfile # Main cluster configuration
|
|
├── Vagrantfile.allinone # Single-node configuration
|
|
├── Vagrantfile.full # Full cluster backup
|
|
├── files/
|
|
│ ├── hosts # Ansible inventory
|
|
│ ├── hosts-allinone # Single-node inventory
|
|
│ ├── key # SSH private key
|
|
│ ├── key.pub # SSH public key
|
|
│ └── *.ldif # LDAP configuration
|
|
├── provision/ # Provisioning scripts
|
|
├── examples/ # Usage examples
|
|
│ ├── authentication/ # Auth configuration
|
|
│ ├── persistent-volumes/ # Storage examples
|
|
│ ├── registry/ # Container registry
|
|
│ ├── s2i/ # Source-to-Image
|
|
│ └── template/ # Application templates
|
|
└── haproxy/ # Load balancer config
|
|
```
|
|
|
|
---
|
|
|
|
## Contributing
|
|
|
|
Feel free to submit issues, feature requests, and pull requests to improve this OKD development environment.
|
|
|
|
## License
|
|
|
|
This project is provided as-is for educational and development purposes.
|