HomeGoogle Cloud “GCP” Local Nexus Images Build.

Google Cloud “GCP” Local Nexus Images Build.

I am using Knicks Oz as a development environment running on Google Cloud VM.
NixOs official website The official GCP does not provide the image. On the Nix Oz wiki page, you can find an article on how to make your own. GCP NixOs image It did not work well in the past but now it seems that the problem of blood image has been solved and now it works well. The Nexus WikiPage article also provides two GCP storage that allows you to find images of older GCPVMs that are no longer actively maintained.

  • nixos-images> gs: // nixos-images
  • nixos-cloud-images> gs: // nixos-images

In “HowTo” below, we’ll cover how you can create your own photos in GCP using Google Cloud VM and store the photos in your Google storage bucket.

Creating a Google Cloud Environment

Before we can get started, you’ll need one. Google Cloud Account.

We will create a dedicated GCP project with VM that we will use to create the latest NixOs images from the latest nixpkgs builds.

Requirements

Before you begin.
In the first step, you will need to install. Google Cloud SDK Depending on the OS or Linux distribution, and please follow the installation instructions.

Gcloud Validation

Once the installation is complete, you’ll need to verify the CLI to access your Google Cloud Resources.

~> gcloud auth login
Go to the following link in your browser:
   https://accounts.google.com/o/oauth2/auth?response_type=code&client_id=32542940657.apps.googleusercontent.com&redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob&scope=openid+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.email+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcloud-platform+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fappengine.admin+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcompute+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Faccounts.reauth&state=skHTaEGrhJSYIzDIwt4phrhSzUm97t&prompt=consent&access_type=offline&code_challenge=EcgLC0aZhpefFkL7k6ep-8lvJ1Og8NgCvs9VllOE5lQ&code_challenge_method=S256

Enter verification code: 

GCP Project Creation

Once verified, you should be able to access all cloud resources in your Google Cloud account.
If you are using your account for the first time, Google will create a default project for your account. We will ignore the default plan and create a dedicated project to create images of NioxOs.

GCP New Project Creation.

We will create a dedicated GCP project. I use the name for the product Modrix Nexus.

~> gcloud projects create mudrii-nixos 
Create in progress for [https://cloudresourcemanager.googleapis.com/v1/projects/mudrii-nixos].
Waiting for [operations/cp.8533728690478532386] to finish...done. Enabling service [cloudapis.googleapis.com] on project [mudrii-nixos]...
Operation "operations/acf.p2-237810657129-fa1f0fa7-3015-46ff-ab52-f92d7bd3f8df" finished successfully.

Get details about the project created:

 ~> gcloud projects describe mudrii-nixos
createTime: '2021-09-06T08:57:16.001Z'
lifecycleState: ACTIVE
name: mudrii-nixos
projectId: mudrii-nixos
projectNumber: '111111111111'
~> gcloud projects list
PROJECT_ID             NAME              PROJECT_NUMBER
mudrii-nixos           mudrii-nixos      111111111111

Configuring Gcloud CLI

Once a new project is created, it’s a good idea to add the newly created project as the default for the Gcloud CLI. Every command executed with Gsutils’ Gcloud will follow the newly created plan.

~> gcloud config set project mudrii-nixos
Updated property [core/project].

Creating project billing

Another step is needed to make the GCP project workable. We will need to include billing in the plan.

List of available billing accounts.

~> gcloud alpha billing accounts list
ACCOUNT_ID            NAME                OPEN   MASTER_ACCOUNT_ID
ZZZZZ-ZZZZZZZ-ZZZZZZ  MyBill              True

Linking a Billing Account to a Project

~> gcloud alpha billing accounts projects link mudrii-nixos --billing-account=ZZZZZ-ZZZZZZZ-ZZZZZZ 
billingAccountName: billingAccounts/ZZZZZ-ZZZZZZZ-ZZZZZZ 
billingEnabled: true
name: projects/mudrii-nixos/billingInfo
projectId: mudrii-nixos

Billing Account Association Confirmation

~> gcloud alpha billing accounts projects list --billing-account=ZZZZZZ-ZZZZZZ-ZZZZZZ
PROJECT_ID             BILLING_ACCOUNT_ID    BILLING_ENABLED
mudrii-nixos           ZZZZZ-ZZZZZZZ-ZZZZZZ  True

Create and configure Google storage buckets.

The next step is to create a Google storage bucket to store the generated Nexus GCPVM images.

~> gsutil mb gs://nixos-images-gcp
gsutil mb gs://nixos-images-gcp


~> gsutil du -s -h gs://nixos-images-gcp

I decided to give external users reading access to the images I created. If you do not intend to share the created images, you can skip the step below.

~> gsutil iam ch allUsers:objectViewer gs://nixos-images-gcp

Creating and configuring GCP networking.

By default, Google will create a global VPC and firewall rules for each region after becoming a subnet and a project.

To clean it up, I decided to create a separate network stack with subnet and firewall to better manage resources.

Before creating a network, it is a good idea to remove all automatic default network VPCs, subnets and firewalls.

List of available network resources.

List VPC:

~> gcloud compute networks list
NAME     SUBNET_MODE  BGP_ROUTING_MODE  IPV4_RANGE  GATEWAY_IPV4
default  AUTO         REGIONAL

Listing subnets.

~> gcloud compute networks subnets list
NAME     REGION                   NETWORK  RANGE
default  us-central1              default  10.128.0.0/20
default  europe-west1             default  10.132.0.0/20
default  us-west1                 default  10.138.0.0/20
default  asia-east1               default  10.140.0.0/20
default  us-east1                 default  10.142.0.0/20
default  asia-northeast1          default  10.146.0.0/20
default  asia-southeast1          default  10.148.0.0/20
default  us-east4                 default  10.150.0.0/20
default  australia-southeast1     default  10.152.0.0/20
default  europe-west2             default  10.154.0.0/20
default  europe-west3             default  10.156.0.0/20
default  southamerica-east1       default  10.158.0.0/20
default  asia-south1              default  10.160.0.0/20
default  northamerica-northeast1  default  10.162.0.0/20
default  europe-west4             default  10.164.0.0/20
default  europe-north1            default  10.166.0.0/20
default  us-west2                 default  10.168.0.0/20
default  asia-east2               default  10.170.0.0/20
default  europe-west6             default  10.172.0.0/20
default  asia-northeast2          default  10.174.0.0/20
default  asia-northeast3          default  10.178.0.0/20
default  us-west3                 default  10.180.0.0/20
default  us-west4                 default  10.182.0.0/20
default  asia-southeast2          default  10.184.0.0/20
default  europe-central2          default  10.186.0.0/20
default  northamerica-northeast2  default  10.188.0.0/20
default  asia-south2              default  10.190.0.0/20
default  australia-southeast2     default  10.192.0.0/20

List of firewalls.

~> gcloud compute firewall-rules list 
NAME                    NETWORK  DIRECTION  PRIORITY  ALLOW                         DENY  DISABLED
default-allow-icmp      default  INGRESS    65534     icmp                                False
default-allow-internal  default  INGRESS    65534     tcp:0-65535,udp:0-65535,icmp        False
default-allow-rdp       default  INGRESS    65534     tcp:3389                            False
default-allow-ssh       default  INGRESS    65534     tcp:22                              False

To show all fields of the firewall, please show in JSON format: --format=json
To show all fields in table format, please see the examples in --help.

Remove the default created network resource.

Before removing the VPC, we need to remove the firewall first, and once all the rules of the firewall are removed, we can remove the default VPC.

Remove the default firewall.

~> gcloud compute firewall-rules delete default-allow-internal --quiet
Deleted [https://www.googleapis.com/compute/v1/projects/mudrii-nixos/global/firewalls/default-allow-internal].


~> gcloud compute firewall-rules delete default-allow-icmp --quiet 
Deleted [https://www.googleapis.com/compute/v1/projects/mudrii-nixos/global/firewalls/default-allow-icmp].


~> gcloud compute firewall-rules delete default-allow-rdp --quiet
Deleted [https://www.googleapis.com/compute/v1/projects/mudrii-nixos/global/firewalls/default-allow-rdp].


~> gcloud compute firewall-rules delete default-allow-ssh --quiet
Deleted [https://www.googleapis.com/compute/v1/projects/mudrii-nixos/global/firewalls/default-allow-ssh].

Remove default VPC

~> gcloud compute networks delete default --quiet
Deleted [https://www.googleapis.com/compute/v1/projects/mudrii-nixos/global/networks/default].

Creating networking resources

We’ve cleared all the default networks that are stuck, and now we can create a new one.

Creating a VPC:

Now, we are ready to create a custom VPC.

~> gcloud compute networks create vpc-nixos --subnet-mode=custom 
Created [https://www.googleapis.com/compute/v1/projects/mudrii-nixos/global/networks/vpc-nixos].
NAME       SUBNET_MODE  BGP_ROUTING_MODE  IPV4_RANGE  GATEWAY_IPV4
vpc-nixos  CUSTOM       REGIONAL

Instances on this network will not be reachable until firewall rules
are created. As an example, you can allow all internal traffic between
instances as well as SSH, RDP, and ICMP by running:

Creating a custom subnet.

In the example below, I use a narrow IP range in the Asia Southeast 1 “Singapore” region. Since I will run an example that will produce Nexus images, I have no intention of doing anything else.

~> gcloud compute networks subnets create subnet-sg --network=vpc-nixos --range=192.168.1.0/24 --region asia-southeast1
Created [https://www.googleapis.com/compute/v1/projects/mudrii-nixos/regions/asia-southeast1/subnetworks/subnet-sg].
NAME         REGION           NETWORK    RANGE
subnet-sg  asia-southeast1  vpc-nixos  192.168.1.0/24

Create a firewall role to allow SSH.

To connect to remote VM on ssh, we need to open the firewall on port 22.

~> gcloud compute --project=mudrii-nixos firewall-rules create allow-ssh --direction=INGRESS --priority=1000 --network=vpc-nixos --action=ALLOW --rules=tcp:22 --target-tags=allow-ssh
Creating firewall...⠹Created [https://www.googleapis.com/compute/v1/projects/mudrii-nixos/global/firewalls/allow-ssh].
Creating firewall...done.
NAME       NETWORK    DIRECTION  PRIORITY  ALLOW   DENY  DISABLED
allow-ssh  vpc-nixos  INGRESS    1000      tcp:22        False

Adding an image to your VM images repository to be able to create NixOs VM in GCP

To create a VM with NixOS installed, we need to have a VM image in our GCP image repositories.

In the example below, I’m using one of the pictures I made earlier. gs://nixos-images-gcp/

Note: Make sure you provide your project ID. --project=mudrii-nixos

~> gcloud compute images create nixos-21-05-2873 
--source-uri=gs://nixos-images-gcp/nixos-image-21.05.2873.6bfe71f2a4e-x86_64-linux.raw.tar.gz 
--description=nixos-image-21.05.2873.6bfe71f2a4e 
--family=nixos 
--project=mudrii-nixos

List the available photos in your photo archive.

 ~> gcloud compute images list | grep nixos
nixos-image-21-05-2873-6bfe71f2a4e-x86-64-linux       mudrii-nixos         nixos-image-21-05                             READY

Creating NixOS VM

Our network is stuck. We can create a dedicated VM to create Nixus GCP images.

Here are a few points:

  • The VM will be built in the region where we set up our subnet. --zone=asia-southeast1-b
  • --machine-type=n2d-standard-4 VM AMD EMYC 4 vCPU and 16 GB RAM.
  • --metadata=enable-oslogin=true Allows you to log in to VM with your Gcloud account.
  • --tags=allow-ssh The tag was added to the VM to allow open port SSH which we created in the firewall.
  • --boot-disk-size=60GB Disk size you can choose low capacity I also like 30 works.
  • --boot-disk-type=pd-ssd This is the fastest option. Local SSD will significantly speed up image generation.
~> gcloud beta compute --project=mudrii-nixos instances create nixos-base --zone=asia-southeast1-b --machine-type=n2d-standard-4 --subnet=subnet-sg --network-tier=PREMIUM --metadata=enable-oslogin=true --tags=allow-ssh --image=nixos-21-05-2873 --image-project=mudrii-nixos --boot-disk-size=60GB --boot-disk-type=pd-ssd --boot-disk-device-name=nixos-base
Created [https://www.googleapis.com/compute/beta/projects/mudrii-nixos/zones/asia-southeast1-b/instances/nixos-base].
WARNING: Some requests generated warnings:
 - Disk size: '60 GB' is larger than image size: '3 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.
 
NAME        ZONE               MACHINE_TYPE    PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP     STATUS
nixos-base  asia-southeast1-b  n2d-standard-4               192.168.1.2  35.188.151.168  RUNNING

SSH in the remote.

We can ssh directly with the newly created VM. gcloud Command:

~> gcloud compute ssh --project=mudrii-nixos nixos-base

NixOS image building

Once SSH enters the newly created NixOs Google VM, we can begin preparations to create a NixoS CGP image.

Verify from VM to Google Cloud.

First, we need to be rooted.

~> sudo -i 

We need access to Google SDK to upload newly created images to the Google Bucket created in the previous steps. To verify, we need the Google SDK, and instead of installing, we will use. nix-shell.

[root@nixos-base:~]# nix-shell -p google-cloud-sdk git

Once we have access to Gcloud Binary, we can sign in to Google Cloud.

[nix-shell:~]# gcloud auth login


[nix-shell:~]# gcloud projects list
PROJECT_ID    NAME          PROJECT_NUMBER
mudrii-nixos  mudrii-nixos  111111111111


[nix-shell:~]# gcloud config set project mudrii-nixos
Updated property [core/project].


[nix-shell:~]# gcloud config configurations list
NAME     IS_ACTIVE  ACCOUNT            PROJECT       COMPUTE_DEFAULT_ZONE  COMPUTE_DEFAULT_REGION
default  True       my_email@gmail.com  mudrii-nixos

To verify, you can check if you have access to your Gcloud bucket created in the previous step.

Note: Be sure to add. BOTO_CONFIG=/dev/null Or you may encounter an error while creating the image.

[nix-shell:~]# export BOTO_CONFIG=/dev/null
[nix-shell:~]# gsutil ls -l gs://nixos-images-gcp

Update NixOS to the latest version.

To create an image with the latest stable NixOs, it is necessary to update the current version with the latest version.

Check out the current NixOs version and the Linux kernel.

[nix-shell:~]# nixos-version
21.05.3001.12eb1d16ae3 (Okapi)


[nix-shell:~]# uname -a
Linux nixos-base.asia-southeast1-b.c.mudrii-nixos.internal 5.10.62 #1-NixOS SMP Fri Sep 3 08:09:31 UTC 2021 x86_64 GNU/Linux

Updating NixOs to the latest version.

Update Nix Channels first:
[nix-shell:~]# nix-channel --list
nixos https://nixos.org/channels/nixos-21.05


[nix-shell:~]# nix-channel --update 
unpacking channels...
created 1 symlinks in user environment
Update NixOS first:
[nix-shell:~]# nixos-rebuild switch
building Nix...
building the system configuration...
these derivations will be built:
...
..
.

[nix-shell:~]# nixos-version
21.05.3021.8b0b81dab17 (Okapi)

Or, if the version is significantly older, run the following:

[nix-shell:~]# nixos-rebuild switch --upgrade

Knox Oz Cleanup

Before attempting to create an image, it is a good idea to clean up and collect garbage on current updates and repair any inconsistencies that may lead to image failure.

[nix-shell:~]# nix-collect-garbage -d
removing old generations of profile /nix/var/nix/profiles/system
removing generation 4
removing old generations of profile /nix/var/nix/profiles/per-user/root/channels
removing generation 5
finding garbage collector roots...
removing stale link from 
...
..
.
deleting '/nix/store/trash'
deleting unused links...
note: currently hard linking saves 21.60 MiB
167 store paths deleted, 1129.80 MiB freed




[nix-shell:~]# nix-store --gc
finding garbage collector roots...
deleting garbage...
deleting '/nix/store/trash'
deleting unused links...
note: currently hard linking saves 21.60 MiB
0 store paths deleted, 0.00 MiB freed




[nix-shell:~]# nix-store --repair --verify --check-contents
reading the Nix store...
checking path existence...
checking hashes...
path '/nix/store/kacsvbh8qjl28izy5g7a8p96z6xdvnik-google-cloud-sdk-340.0.0' was modified! expected hash 'sha256:0dvxzzklaswx0d2svx0nzjilqfmgd2dxffi7hcbz89p7r6w1jab0', got 'sha256:1wpsb25jajbqvsw29jy073w0cr994005xb22pb9p8mkinn9vx8hp'
copying path '/nix/store/kacsvbh8qjl28izy5g7a8p96z6xdvnik-google-cloud-sdk-340.0.0' from 'https://cache.nixos.org'...




[nix-shell:~]# nix-store --optimise
430.37 MiB freed by hard-linking 47107 files

NixOs GCP image build.

To create a GCP Nixus image we need to clone the Nix PK GS repository where the blood scripts are located:

[nix-shell:~]# git clone https://github.com/NixOS/nixpkgs.git --depth 1
Cloning into 'nixpkgs'...
remote: Enumerating objects: 45763, done.
remote: Counting objects: 100% (45763/45763), done.
remote: Compressing objects: 100% (29848/29848), done.
remote: Total 45763 (delta 1593), reused 38844 (delta 1293), pack-reused 0
Receiving objects: 100% (45763/45763), 30.46 MiB | 14.19 MiB/s, done.
Resolving deltas: 100% (1593/1593), done.
Updating files: 100% (27800/27800), done.

The last step is to run a script that will create a GCP nixos image based on the basic system version.

Note: Be sure to name your GCP storage bucket. BUCKET_NAME=

[nix-shell:~]# BUCKET_NAME=nixos-images-gcp nixpkgs/nixos/maintainers/scripts/gce/create-gce.sh
these paths will be fetched (0.05 MiB download, 0.28 MiB unpacked):
  /nix/store/p5lnl4zr45n7mf9kz9w8yz3rqh001b5c-bash-interactive-4.4-p23-dev
copying path '/nix/store/p5lnl4zr45n7mf9kz9w8yz3rqh001b5c-bash-interactive-4.4-p23-dev' from 'https://cache.nixos.org'...
...
..
.
/nix/store/ii2h0jqwfzmzdc6lxyfmg4ia5726r6g6-google-compute-image
gs://nixos-images-gcp/nixos-image-21.05.3021.8b0b81dab17-x86_64-linux.raw.tar.gz

NixOs Image Blood Verification.

Once the image is created and uploaded to the Google Storage Bucket and to the image repository, we can verify by doing the following:

[nix-shell:~]# gsutil ls -l gs://nixos-images-gcp
 434150003  2021-09-12T03:18:26Z  gs://nixos-images-gcp/nixos-image-21.05.3021.8b0b81dab17-x86_64-linux.raw.tar.gz

Confirm Image Archive:

[nix-shell:~]# gcloud compute images list | grep nixos
nixos-image-21-05-3021-8b0b81dab17-x86-64-linux       mudrii-nixos         nixos-image-21-05                             READY

Cleaning after construction.

It is a good idea to remove the nixpkgs repository to minimize space and speed on the next build and stop VM to reduce the cost on your cloud bill.

[nix-shell:~]# ls -la
total 44
drwx------  8 root root 4096 Sep 13 11:02 .
drwxr-xr-x 17 root root 4096 Sep  6 10:55 ..
-rw-------  1 root root 2538 Sep 10 10:27 .bash_history
drwxr-xr-x  3 root root 4096 Sep  6 11:07 .cache
drwxr-xr-x  3 root root 4096 Sep  6 11:07 .config
lrwxrwxrwx  1 root root   64 Sep 13 11:02 gce -> /nix/store/ii2h0jqwfzmzdc6lxyfmg4ia5726r6g6-google-compute-image
drwxr-xr-x  3 root root 4096 Sep  6 11:48 .gsutil
-rw-r--r--  1 root root   45 Aug 25 05:05 .nix-channels
drwx------  2 root root 4096 Sep 13 10:30 .nix-defexpr
drwxr-xr-x  9 root root 4096 Sep 13 10:55 nixpkgs
drwx------  2 root root 4096 Sep  6 10:55 .ssh


[nix-shell:~]# rm -rf ~/nixpkgs


[nix-shell:~]# sudo shutdown now
Connection to 35.198.250.170 closed by remote host.
Connection to 35.198.250.170 closed.
ERROR: (gcloud.compute.ssh) [/run/current-system/sw/bin/ssh] exited with return code [255].

Finished

Next time, when you want to create a new VM on Google Cloud, you can describe your latest Nixus image in custom images. Be sure to add. --metadata=enable-oslogin=true Allow you to log in to VM.

.