feature: Adds README (oops).

This commit is contained in:
greysoh 2024-07-14 12:20:50 -04:00
parent c8d78b5e00
commit b02391f230
Signed by: imterah
GPG key ID: 8FA7DD57BA6CEA37

View file

@ -1,2 +1,38 @@
# KittehCluster
This is my Kubernetes clustered computing setup, based on Proxmox VE and NixOS.
This is my (work in progress, deployed but nothing production running on it *yet*) Kubernetes clustered computing setup, based on Proxmox VE and NixOS.
Currently, I cannot recommend that you use this setup in production yet. I have to delete and recreate my VMs multiple times a day, until I fix everything.
## Prerequisites
- An x86_64 computer with virtualization enabled, running NixOS
- A cluster of computers running Proxmox, with your SSH keys copied to them. These should (not required, but *highly* recommended) be connected together in Proxmox using the cluster feature.
- Cluster hypervisor's IPs next to eachother (ex. node 1's Proxmox is `192.168.0.20`, node 2's is `192.168.0.21`)
- Patience (will take a while, and may test it)
- `kubectl`, and `helm` installed on your local computer.
## Adding nodes
Copy `kitteh-node-2`, to `kitteh-node-X`, where `X` is the server number. Change the hostname to correspond to each clustered computer (ex. 3rd computer's k3s agent is `kitteh-node-3-k3s-agent`)
## VM Setup
1. First, you'll need to fork this repository, and `git clone` it down.
2. Copy `secrets.example.nix` to `secrets.nix`.
3. Change `services.k3s.token` to be a unique token (i.e. using `uuidgen`, `head -c 500 /dev/random | sha1sum | cut -d " " -f 1`, etc)
4. Change `users.users.clusteradm.openssh.authorizedKeys.keys` to have your SSH key(s) in there.
5. Then, run `./buildall.sh`, to build all the virtual machines. This may take a long time, depending on your hardware! On a 2015 MacBook Air, this took 30 minutes. Make some tea while you wait!
6. Finally, run `BASE_IP=your_base_ip_here ./upload.sh -i -d`, with `BASE_IP` being the first IP for your Proxmox cluster.
7. Set all VMs to auto-start, then turn them all on, starting with the first node's `k3s-server`.
8. You can now connect using your SSH key to any of the nodes with the user `clusteradm`. The default password is `1234`. Be sure to change this!
## Kubernetes setup
1. SSH into any of the nodes. (i.e. `ssh clusteradm@kitteh-node-2-k3s-server`)
2. As root, grab `/etc/rancher/k3s/k3s.yaml`, and copy it to wherever you store your k3s configurations (on macOS, this is `~/.kube/config`)
## Updating (TODO)
In NixOS, instead of `apt update; apt upgrade -y`, `pacman -Syu --noconfirm`, or other systems, you need to "rebuild" the system.
There is a work in progress of this system (see `kittehclean`'s Git downloader), but it is not done yet.
## Custom cluster setup / Forking
1. First, fork this Git repository if you haven't already.
2. If you want to change the folder names, rename the folders (i.e. kitteh-node-* to whatever-*), and change `buildall.sh`'s for loop to be `whatever-*/*`, for example.
3. If you want to change the hostname, change them all. Be sure to change `commons.agent.nix` and `commons.server.nix` to correspond to the new `kitteh-node-1-k3s-server`'s name!
4. In `commons.nix`, either remove `kittehclean` (not recommended unless you're using a private Git repository), or change the git repository it pulls down from (i.e. change `https://git.hofers.cloud/greysoh/kittehcluster` to `https://github.com/contoso/k3s-cluster`).
5. (optional) Rename `kittehclean` and change the description.
## Troubleshooting
- I can't login via SSH!
- Have you copied your SSH keys to the `clusteradm` user? Try copying your keys on another computer (or the VM console) if you got a new one, for example (in the `~/.ssh/authorized_keys` on each VM)
- Additionally, password authentication is disabled!