I am using Knicks Oz as a development environment running on Google Cloud VM.
NixOs official website The official GCP does not provide the image. On the Nix Oz wiki page, you can find an article on how to make your own. GCP NixOs image It did not work well in the past but now it seems that the problem of blood image has been solved and now it works well. The Nexus WikiPage article also provides two GCP storage that allows you to find images of older GCPVMs that are no longer actively maintained.
- nixos-images> gs: // nixos-images
- nixos-cloud-images> gs: // nixos-images
In “HowTo” below, we’ll cover how you can create your own photos in GCP using Google Cloud VM and store the photos in your Google storage bucket.
Creating a Google Cloud Environment
Before we can get started, you’ll need one. Google Cloud Account.
We will create a dedicated GCP project with VM that we will use to create the latest NixOs images from the latest nixpkgs builds.
Requirements
Before you begin.
In the first step, you will need to install. Google Cloud SDK Depending on the OS or Linux distribution, and please follow the installation instructions.
Gcloud Validation
Once the installation is complete, you’ll need to verify the CLI to access your Google Cloud Resources.
~> gcloud auth login
Go to the following link in your browser:
https://accounts.google.com/o/oauth2/auth?response_type=code&client_id=32542940657.apps.googleusercontent.com&redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob&scope=openid+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.email+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcloud-platform+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fappengine.admin+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcompute+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Faccounts.reauth&state=skHTaEGrhJSYIzDIwt4phrhSzUm97t&prompt=consent&access_type=offline&code_challenge=EcgLC0aZhpefFkL7k6ep-8lvJ1Og8NgCvs9VllOE5lQ&code_challenge_method=S256
Enter verification code:
GCP Project Creation
Once verified, you should be able to access all cloud resources in your Google Cloud account.
If you are using your account for the first time, Google will create a default project for your account. We will ignore the default plan and create a dedicated project to create images of NioxOs.
GCP New Project Creation.
We will create a dedicated GCP project. I use the name for the product Modrix Nexus.
~> gcloud projects create mudrii-nixos
Create in progress for [https://cloudresourcemanager.googleapis.com/v1/projects/mudrii-nixos].
Waiting for [operations/cp.8533728690478532386] to finish...done. Enabling service [cloudapis.googleapis.com] on project [mudrii-nixos]...
Operation "operations/acf.p2-237810657129-fa1f0fa7-3015-46ff-ab52-f92d7bd3f8df" finished successfully.
Get details about the project created:
~> gcloud projects describe mudrii-nixos
createTime: '2021-09-06T08:57:16.001Z'
lifecycleState: ACTIVE
name: mudrii-nixos
projectId: mudrii-nixos
projectNumber: '111111111111'
~> gcloud projects list
PROJECT_ID NAME PROJECT_NUMBER
mudrii-nixos mudrii-nixos 111111111111
Configuring Gcloud CLI
Once a new project is created, it’s a good idea to add the newly created project as the default for the Gcloud CLI. Every command executed with Gsutils’ Gcloud will follow the newly created plan.
~> gcloud config set project mudrii-nixos
Updated property [core/project].
Creating project billing
Another step is needed to make the GCP project workable. We will need to include billing in the plan.
List of available billing accounts.
~> gcloud alpha billing accounts list
ACCOUNT_ID NAME OPEN MASTER_ACCOUNT_ID
ZZZZZ-ZZZZZZZ-ZZZZZZ MyBill True
Linking a Billing Account to a Project
~> gcloud alpha billing accounts projects link mudrii-nixos --billing-account=ZZZZZ-ZZZZZZZ-ZZZZZZ
billingAccountName: billingAccounts/ZZZZZ-ZZZZZZZ-ZZZZZZ
billingEnabled: true
name: projects/mudrii-nixos/billingInfo
projectId: mudrii-nixos
Billing Account Association Confirmation
~> gcloud alpha billing accounts projects list --billing-account=ZZZZZZ-ZZZZZZ-ZZZZZZ
PROJECT_ID BILLING_ACCOUNT_ID BILLING_ENABLED
mudrii-nixos ZZZZZ-ZZZZZZZ-ZZZZZZ True
Create and configure Google storage buckets.
The next step is to create a Google storage bucket to store the generated Nexus GCPVM images.
~> gsutil mb gs://nixos-images-gcp gsutil mb gs://nixos-images-gcp~> gsutil du -s -h gs://nixos-images-gcp
I decided to give external users reading access to the images I created. If you do not intend to share the created images, you can skip the step below.
~> gsutil iam ch allUsers:objectViewer gs://nixos-images-gcp
Creating and configuring GCP networking.
By default, Google will create a global VPC and firewall rules for each region after becoming a subnet and a project.
To clean it up, I decided to create a separate network stack with subnet and firewall to better manage resources.
Before creating a network, it is a good idea to remove all automatic default network VPCs, subnets and firewalls.
List of available network resources.
List VPC:
~> gcloud compute networks list
NAME SUBNET_MODE BGP_ROUTING_MODE IPV4_RANGE GATEWAY_IPV4
default AUTO REGIONAL
Listing subnets.
~> gcloud compute networks subnets list
NAME REGION NETWORK RANGE
default us-central1 default 10.128.0.0/20
default europe-west1 default 10.132.0.0/20
default us-west1 default 10.138.0.0/20
default asia-east1 default 10.140.0.0/20
default us-east1 default 10.142.0.0/20
default asia-northeast1 default 10.146.0.0/20
default asia-southeast1 default 10.148.0.0/20
default us-east4 default 10.150.0.0/20
default australia-southeast1 default 10.152.0.0/20
default europe-west2 default 10.154.0.0/20
default europe-west3 default 10.156.0.0/20
default southamerica-east1 default 10.158.0.0/20
default asia-south1 default 10.160.0.0/20
default northamerica-northeast1 default 10.162.0.0/20
default europe-west4 default 10.164.0.0/20
default europe-north1 default 10.166.0.0/20
default us-west2 default 10.168.0.0/20
default asia-east2 default 10.170.0.0/20
default europe-west6 default 10.172.0.0/20
default asia-northeast2 default 10.174.0.0/20
default asia-northeast3 default 10.178.0.0/20
default us-west3 default 10.180.0.0/20
default us-west4 default 10.182.0.0/20
default asia-southeast2 default 10.184.0.0/20
default europe-central2 default 10.186.0.0/20
default northamerica-northeast2 default 10.188.0.0/20
default asia-south2 default 10.190.0.0/20
default australia-southeast2 default 10.192.0.0/20
List of firewalls.
~> gcloud compute firewall-rules list
NAME NETWORK DIRECTION PRIORITY ALLOW DENY DISABLED
default-allow-icmp default INGRESS 65534 icmp False
default-allow-internal default INGRESS 65534 tcp:0-65535,udp:0-65535,icmp False
default-allow-rdp default INGRESS 65534 tcp:3389 False
default-allow-ssh default INGRESS 65534 tcp:22 False
To show all fields of the firewall, please show in JSON format: --format=json
To show all fields in table format, please see the examples in --help.
Remove the default created network resource.
Before removing the VPC, we need to remove the firewall first, and once all the rules of the firewall are removed, we can remove the default VPC.
Remove the default firewall.
~> gcloud compute firewall-rules delete default-allow-internal --quiet Deleted [https://www.googleapis.com/compute/v1/projects/mudrii-nixos/global/firewalls/default-allow-internal].~> gcloud compute firewall-rules delete default-allow-icmp --quiet Deleted [https://www.googleapis.com/compute/v1/projects/mudrii-nixos/global/firewalls/default-allow-icmp].~> gcloud compute firewall-rules delete default-allow-rdp --quiet Deleted [https://www.googleapis.com/compute/v1/projects/mudrii-nixos/global/firewalls/default-allow-rdp].~> gcloud compute firewall-rules delete default-allow-ssh --quiet Deleted [https://www.googleapis.com/compute/v1/projects/mudrii-nixos/global/firewalls/default-allow-ssh].
Remove default VPC
~> gcloud compute networks delete default --quiet
Deleted [https://www.googleapis.com/compute/v1/projects/mudrii-nixos/global/networks/default].
Creating networking resources
We’ve cleared all the default networks that are stuck, and now we can create a new one.
Creating a VPC:
Now, we are ready to create a custom VPC.
~> gcloud compute networks create vpc-nixos --subnet-mode=custom
Created [https://www.googleapis.com/compute/v1/projects/mudrii-nixos/global/networks/vpc-nixos].
NAME SUBNET_MODE BGP_ROUTING_MODE IPV4_RANGE GATEWAY_IPV4
vpc-nixos CUSTOM REGIONAL
Instances on this network will not be reachable until firewall rules
are created. As an example, you can allow all internal traffic between
instances as well as SSH, RDP, and ICMP by running:
Creating a custom subnet.
In the example below, I use a narrow IP range in the Asia Southeast 1 “Singapore” region. Since I will run an example that will produce Nexus images, I have no intention of doing anything else.
~> gcloud compute networks subnets create subnet-sg --network=vpc-nixos --range=192.168.1.0/24 --region asia-southeast1
Created [https://www.googleapis.com/compute/v1/projects/mudrii-nixos/regions/asia-southeast1/subnetworks/subnet-sg].
NAME REGION NETWORK RANGE
subnet-sg asia-southeast1 vpc-nixos 192.168.1.0/24
Create a firewall role to allow SSH.
To connect to remote VM on ssh, we need to open the firewall on port 22.
~> gcloud compute --project=mudrii-nixos firewall-rules create allow-ssh --direction=INGRESS --priority=1000 --network=vpc-nixos --action=ALLOW --rules=tcp:22 --target-tags=allow-ssh
Creating firewall...â ıCreated [https://www.googleapis.com/compute/v1/projects/mudrii-nixos/global/firewalls/allow-ssh].
Creating firewall...done.
NAME NETWORK DIRECTION PRIORITY ALLOW DENY DISABLED
allow-ssh vpc-nixos INGRESS 1000 tcp:22 False
Adding an image to your VM images repository to be able to create NixOs VM in GCP
To create a VM with NixOS installed, we need to have a VM image in our GCP image repositories.
In the example below, I’m using one of the pictures I made earlier. gs://nixos-images-gcp/
Note: Make sure you provide your project ID. --project=mudrii-nixos
~> gcloud compute images create nixos-21-05-2873
--source-uri=gs://nixos-images-gcp/nixos-image-21.05.2873.6bfe71f2a4e-x86_64-linux.raw.tar.gz
--description=nixos-image-21.05.2873.6bfe71f2a4e
--family=nixos
--project=mudrii-nixos
List the available photos in your photo archive.
~> gcloud compute images list | grep nixos
nixos-image-21-05-2873-6bfe71f2a4e-x86-64-linux mudrii-nixos nixos-image-21-05 READY
Creating NixOS VM
Our network is stuck. We can create a dedicated VM to create Nixus GCP images.
Here are a few points:
- The VM will be built in the region where we set up our subnet.
--zone=asia-southeast1-b --machine-type=n2d-standard-4VM AMD EMYC 4 vCPU and 16 GB RAM.--metadata=enable-oslogin=trueAllows you to log in to VM with your Gcloud account.--tags=allow-sshThe tag was added to the VM to allow open port SSH which we created in the firewall.--boot-disk-size=60GBDisk size you can choose low capacity I also like 30 works.--boot-disk-type=pd-ssdThis is the fastest option. Local SSD will significantly speed up image generation.
~> gcloud beta compute --project=mudrii-nixos instances create nixos-base --zone=asia-southeast1-b --machine-type=n2d-standard-4 --subnet=subnet-sg --network-tier=PREMIUM --metadata=enable-oslogin=true --tags=allow-ssh --image=nixos-21-05-2873 --image-project=mudrii-nixos --boot-disk-size=60GB --boot-disk-type=pd-ssd --boot-disk-device-name=nixos-base
Created [https://www.googleapis.com/compute/beta/projects/mudrii-nixos/zones/asia-southeast1-b/instances/nixos-base].
WARNING: Some requests generated warnings:
- Disk size: '60 GB' is larger than image size: '3 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.
NAME ZONE MACHINE_TYPE PREEMPTIBLE INTERNAL_IP EXTERNAL_IP STATUS
nixos-base asia-southeast1-b n2d-standard-4 192.168.1.2 35.188.151.168 RUNNING
SSH in the remote.
We can ssh directly with the newly created VM. gcloud Command:
~> gcloud compute ssh --project=mudrii-nixos nixos-base
NixOS image building
Once SSH enters the newly created NixOs Google VM, we can begin preparations to create a NixoS CGP image.
Verify from VM to Google Cloud.
First, we need to be rooted.
~> sudo -i
We need access to Google SDK to upload newly created images to the Google Bucket created in the previous steps. To verify, we need the Google SDK, and instead of installing, we will use. nix-shell.
[root@nixos-base:~]# nix-shell -p google-cloud-sdk git
Once we have access to Gcloud Binary, we can sign in to Google Cloud.
[nix-shell:~]# gcloud auth login[nix-shell:~]# gcloud projects list PROJECT_ID NAME PROJECT_NUMBER mudrii-nixos mudrii-nixos 111111111111[nix-shell:~]# gcloud config set project mudrii-nixos Updated property [core/project].[nix-shell:~]# gcloud config configurations list NAME IS_ACTIVE ACCOUNT PROJECT COMPUTE_DEFAULT_ZONE COMPUTE_DEFAULT_REGION default True my_email@gmail.com mudrii-nixos
To verify, you can check if you have access to your Gcloud bucket created in the previous step.
Note: Be sure to add. BOTO_CONFIG=/dev/null Or you may encounter an error while creating the image.
[nix-shell:~]# export BOTO_CONFIG=/dev/null
[nix-shell:~]# gsutil ls -l gs://nixos-images-gcp
Update NixOS to the latest version.
To create an image with the latest stable NixOs, it is necessary to update the current version with the latest version.
Check out the current NixOs version and the Linux kernel.
[nix-shell:~]# nixos-version 21.05.3001.12eb1d16ae3 (Okapi)[nix-shell:~]# uname -a Linux nixos-base.asia-southeast1-b.c.mudrii-nixos.internal 5.10.62 #1-NixOS SMP Fri Sep 3 08:09:31 UTC 2021 x86_64 GNU/Linux
Updating NixOs to the latest version.
Update Nix Channels first:
[nix-shell:~]# nix-channel --list nixos https://nixos.org/channels/nixos-21.05[nix-shell:~]# nix-channel --update unpacking channels... created 1 symlinks in user environment
Update NixOS first:
[nix-shell:~]# nixos-rebuild switch
building Nix...
building the system configuration...
these derivations will be built:
...
..
.
[nix-shell:~]# nixos-version
21.05.3021.8b0b81dab17 (Okapi)
Or, if the version is significantly older, run the following:
[nix-shell:~]# nixos-rebuild switch --upgrade
Knox Oz Cleanup
Before attempting to create an image, it is a good idea to clean up and collect garbage on current updates and repair any inconsistencies that may lead to image failure.
[nix-shell:~]# nix-collect-garbage -d removing old generations of profile /nix/var/nix/profiles/system removing generation 4 removing old generations of profile /nix/var/nix/profiles/per-user/root/channels removing generation 5 finding garbage collector roots... removing stale link from ... .. . deleting '/nix/store/trash' deleting unused links... note: currently hard linking saves 21.60 MiB 167 store paths deleted, 1129.80 MiB freed[nix-shell:~]# nix-store --gc finding garbage collector roots... deleting garbage... deleting '/nix/store/trash' deleting unused links... note: currently hard linking saves 21.60 MiB 0 store paths deleted, 0.00 MiB freed[nix-shell:~]# nix-store --repair --verify --check-contents reading the Nix store... checking path existence... checking hashes... path '/nix/store/kacsvbh8qjl28izy5g7a8p96z6xdvnik-google-cloud-sdk-340.0.0' was modified! expected hash 'sha256:0dvxzzklaswx0d2svx0nzjilqfmgd2dxffi7hcbz89p7r6w1jab0', got 'sha256:1wpsb25jajbqvsw29jy073w0cr994005xb22pb9p8mkinn9vx8hp' copying path '/nix/store/kacsvbh8qjl28izy5g7a8p96z6xdvnik-google-cloud-sdk-340.0.0' from 'https://cache.nixos.org'...[nix-shell:~]# nix-store --optimise 430.37 MiB freed by hard-linking 47107 files
NixOs GCP image build.
To create a GCP Nixus image we need to clone the Nix PK GS repository where the blood scripts are located:
[nix-shell:~]# git clone https://github.com/NixOS/nixpkgs.git --depth 1
Cloning into 'nixpkgs'...
remote: Enumerating objects: 45763, done.
remote: Counting objects: 100% (45763/45763), done.
remote: Compressing objects: 100% (29848/29848), done.
remote: Total 45763 (delta 1593), reused 38844 (delta 1293), pack-reused 0
Receiving objects: 100% (45763/45763), 30.46 MiB | 14.19 MiB/s, done.
Resolving deltas: 100% (1593/1593), done.
Updating files: 100% (27800/27800), done.
The last step is to run a script that will create a GCP nixos image based on the basic system version.
Note: Be sure to name your GCP storage bucket. BUCKET_NAME=
[nix-shell:~]# BUCKET_NAME=nixos-images-gcp nixpkgs/nixos/maintainers/scripts/gce/create-gce.sh
these paths will be fetched (0.05 MiB download, 0.28 MiB unpacked):
/nix/store/p5lnl4zr45n7mf9kz9w8yz3rqh001b5c-bash-interactive-4.4-p23-dev
copying path '/nix/store/p5lnl4zr45n7mf9kz9w8yz3rqh001b5c-bash-interactive-4.4-p23-dev' from 'https://cache.nixos.org'...
...
..
.
/nix/store/ii2h0jqwfzmzdc6lxyfmg4ia5726r6g6-google-compute-image
gs://nixos-images-gcp/nixos-image-21.05.3021.8b0b81dab17-x86_64-linux.raw.tar.gz
NixOs Image Blood Verification.
Once the image is created and uploaded to the Google Storage Bucket and to the image repository, we can verify by doing the following:
[nix-shell:~]# gsutil ls -l gs://nixos-images-gcp
434150003 2021-09-12T03:18:26Z gs://nixos-images-gcp/nixos-image-21.05.3021.8b0b81dab17-x86_64-linux.raw.tar.gz
Confirm Image Archive:
[nix-shell:~]# gcloud compute images list | grep nixos
nixos-image-21-05-3021-8b0b81dab17-x86-64-linux mudrii-nixos nixos-image-21-05 READY
Cleaning after construction.
It is a good idea to remove the nixpkgs repository to minimize space and speed on the next build and stop VM to reduce the cost on your cloud bill.
[nix-shell:~]# ls -la total 44 drwx------ 8 root root 4096 Sep 13 11:02 . drwxr-xr-x 17 root root 4096 Sep 6 10:55 .. -rw------- 1 root root 2538 Sep 10 10:27 .bash_history drwxr-xr-x 3 root root 4096 Sep 6 11:07 .cache drwxr-xr-x 3 root root 4096 Sep 6 11:07 .config lrwxrwxrwx 1 root root 64 Sep 13 11:02 gce -> /nix/store/ii2h0jqwfzmzdc6lxyfmg4ia5726r6g6-google-compute-image drwxr-xr-x 3 root root 4096 Sep 6 11:48 .gsutil -rw-r--r-- 1 root root 45 Aug 25 05:05 .nix-channels drwx------ 2 root root 4096 Sep 13 10:30 .nix-defexpr drwxr-xr-x 9 root root 4096 Sep 13 10:55 nixpkgs drwx------ 2 root root 4096 Sep 6 10:55 .ssh[nix-shell:~]# rm -rf ~/nixpkgs[nix-shell:~]# sudo shutdown now Connection to 35.198.250.170 closed by remote host. Connection to 35.198.250.170 closed. ERROR: (gcloud.compute.ssh) [/run/current-system/sw/bin/ssh] exited with return code [255].
Finished
Next time, when you want to create a new VM on Google Cloud, you can describe your latest Nixus image in custom images. Be sure to add. --metadata=enable-oslogin=true Allow you to log in to VM.
.