Unified cli

.gitignore

@@ -9,4 +9,10 @@ __pycache__
 *.pyc
 *.pyo
 *.pdf
-*.temp
+*.temp
+*.egg-info
+/.git
+
+/build
+/dist
+/venv

README.md

@@ -1,53 +1,38 @@
-# What?
+# Stiefelsystem
 
-Allows you to boot your (or anyone's really) desktop PC with the exact same
-hard disk that your laptop is running from, basically exactly as if you'd
-unscrew the SSD from your laptop and plug it into the desktop.
+The *Stiefelsystem* allows you to use your computer as a **bootable** storage device **for another computer**, you just need a **network connection** between both.
 
-The data transfer to the hard disk goes over a network link.
+For proper functionality your system has to be compatible with both devices seamlessly, hence this tool is optimized for **Linux**.
 
-# Why?
+Unfortunately, it takes longer than 3 seconds to unscrew the SSD of a Laptop and connect it to the desktop PC, so we wrote this software :smile_cat:
 
-I don't like maintaining multiple operating systems and associated home
-folders.
+Needed components:
+- A computer serving a storage device to to boot from
+- Another computer whose CPU/GPU you actually want to use
+- A network connection between both
+- A USB stick to bootstrap your other computer's startup (no PXE yet)
 
-On the other hand, I like being able to simply use the comfort and power of a
-desktop PC if I encounter one.
 
-Unfortunately, on my laptop it takes longer than 3 seconds to unscrew the SSD,
-so I wrote this.
+## How?
 
-# How?
+Instead of unscrewing your SSD of a Laptop and putting it in a Desktop computer to start it there, the *Stiefelsystem* boots up the same system over network while fetching/storing all system files from the Laptop.
+Since your system is really running on the desktop, you can of course still access an additional storage device and other hardware installed in the Desktop computer (VR Headsets, ...).
 
-The laptop acts as the server.
-It first provides the initrd/kernel via HTTP,
-and then the entire block device for its main disk via NBD.
-A dedicated high-speed network link is recommended for this (henceforth referred to as: stiefellink).
+It works like this: The "server system" (your Laptop) provides the operating system kernel and bootstrapping system over the network to the "client system" (your Desktop computer).
+The bootstrapping system on the client then connects to the server again and mapps the whole storage device (your SSD, mapped with NBD), and then mounts it as root filesystem.
 
-I'm using a RTL8156-based 10/100/1G/2.5G USB3.2 NIC on both sides, in a point-to-point topology.
-The OS cannot be running on the laptop while it serves the disk, for obvious reasons.
-Thus, when the stiefellink NIC is detected by the stiefel-autokexec service, the laptop reboots into
-a custom ramdisk which acts as the stiefelsystem server that provides the aforementioned services.
-Configuration (IP addresses, block device identifiers, ...) is passed to this server through its kernel cmdline.
-The stiefelsystem server will set up the network on stiefellink and wait for requests.
+Now you have one computer serving as network disk, the other having that network disk mounted and running on it.
 
-On the desktop PC, a minimal stiefelsystem client ramdisk is booted from a USB flash drive;
-again, the configuration comes from the kernel cmdline.
-Apart from the cmdline, the ramdisk is actually identical to the server ramdisk.
-The steifelsystem client will search for the server on all of its network interfaces until it receives
-a correct reply. Once it does, it requests the kernel and initrd that it shall boot, and kexec's into them.
-The target system will use a nbd hook in its own initrd to mount the root partition, then boot as usual.
 
-Authentication, encryption and MITM protection happens through a shared symmetric key and AES-EAX.
-Your nbd connection itself is unencrypted and unauthenticated, so I strongly recommend a
-point-to-point connection and not enabling IP forwarding.
+## Communication Flow
+
 
 ```
 time
 |
 |
 | laptop computer                           desktop computer
-| regular OS                            boot from usb stick or PXE
+| regular OS                            boot live system from USB or PXE
 |    |                                            |
 | autokexec service                               |
 v    |              discovery message             |
@@ -72,6 +57,9 @@ v    |              discovery message             |
      v                                            v
 ```
 
+More information can be found in our [more detailed documentation](doc/procedure.md).
+
+
 # How to?
 
 ## Dependencies
@@ -83,66 +71,54 @@ v    |              discovery message             |
 
 ## Setup
 
-Basic steps (commands below):
-
-- You create a Debian-based OS image which is booted on client and server
-- This image is flashed on an USB drive, which is used to boot the client
-- The same system is kexec'd on your server to serve the root disk
+Gist:
 
+- We create a debian based boostrapping system image
+- We flash this image on an USB drive, which is used to boot the client
+- The same boostrapping image is kexec'd on your server to serve the root disk over network
 
-The scripts in this repo automate all of those task (apart from the thinking...)
+Steps:
+- visit the config file, but all defaults should be good to go
+  - the suitable defaults should be set by your Linux distribution in this file!
+- `sudo stiefelctl update`: update the bootstrap image
+- `sudo stiefelctl create-usbdrive /dev/sdxxx`: flash client boot usb thumbdrive with bootstrap image
+- `sudo stiefelctl server`: wait until client connects to serve disks
+  - or, enable/run `stiefelsystem.service` which just runs `stiefelctl server`
 
-- Make shure you have all dependencies installed
-  * `cp config-example.yaml config.yaml` and edit it to your wishes.
-  * Select the modules that are appropriate for your system.
-- create the stiefelsystem ramdisk (for use by server and client)
-  * `sudo ./create-initrd` prepares the debian-based initrd, as a folder and as an archive
-  * You can check out the initrd with `sudo ./test-nspawn`
-  * You can check out server and client interactions with `sudo ./test-qemu server` and `sudo ./test-qemu client`
+## Development
 
-- Setup the `stiefel-autokexec` service on the laptop (and provide it with the ramdisk and config) and setup the nbd rootfs hook in your initfs on your OS
-  * `sudo ./setup-server-os` sets up your system, asking for permission for every operation. It sets up:
-    * The `stiefel-autokexec.service`
-    * Initrd hooks that can mount your root disk from the network
-    * A network manager rule to disable control of the network partition network device
-- Create a USB boot drive
-  * `sudo ./setup-client-usbdrive /dev/sdxxx` creates the usb drive
+- `stiefelctl test-nspawn` to test the stiefelOS image
+- `stiefelctl test-qemu <client|server>` to test client-server interactions with virtual machines
 
-- To reset the AES key, run `rm aes-key` (newly created ramdisks won't work with older ones)
+- To your secret key, remove `aes-key` at the location specified in the config.
 
 
 # Why don't you use X in the tech stack?
 
-I tried X and it sucks.
+We tried X and it sucks.
 
 (for some values of X, including but not limited to:)
 
 - iSCSI (super-slow compared to `nbd`, and overly complicated)
-- PXE (doesn't support my 2.5GBaseT USB NIC)
+- PXE (unreliable, USB sticks worked better, but we may revisit :)
 
 # Why don't you use Y in the tech stack?
 
 We'd really like to use Y, but you haven't implemented support yet.
 
-# Things to improve
-
-## Creation scripts
 
-- Unify create-initrd-nspawn and create-initrd-cpio
-- Allow skipping some of the more time-consuming parts of create-initrd-nspawn
-- setup-client-usbdrive: add a script to launch the client script in any linux's userland
+# Things to improve
 
 ## Network setup
 
 - Allow enabling jumbo frames (8192 bytes?) for higher throughput with bigger files
 - Run a DHCP server on the server (possibly also with PXE support) and support DHCP config on the client
-- Allow server to request earlyboot crypto passphrase from the client in its / HTTP GET answer
 
-## Client script
+## StiefelOS Client
 
 - Search for the server on all network interfaces in parallel, with multiprocessing and network namespaces
 
-## Server script
+## StiefelOS Server
 
 - Produce warning beeps if not on AC power, especially when battery is low
 - Re-setup interfaces as they disappear/appear