Making the windows terminal great again

Once again my employeer forces me to work from an inferior platform, so what are my options to get the best possible working environment? Heres some tips to what I’ve done so far.

The windows terminal is one of the better things produced by Microsoft the last couple of years. I have already written a bit about it before, so search for windows terminal, and you will get the other articles. Even though the terminal is nice, there are some minor annoyances, when you are used to work in a bash environment.

  1. No nice “inline” editor
  2. Does not obey ctrl-d to exit the session
  3. Does not have tab completion on entries in the ssh config file

To fix these three annoyances, we need to download vim from www.vim.org, install it and the rest is done creating and configuring a powershell profile, lets dive into it.

Not only do I have to work from a windows machine (my department does primarily FOSS solutions) I also don’t have any elevated permissions on my machine. IT proffesionlas are treated just the same as office workers šŸ™

This means I cannot install any software, besides whats picked by the Windows admins. Luckily vim is not picky, and you can install it in your own home folder. So lauch the installer and change the install path accordingly.

When vim is installed, all we need to do is to create a powershell profile file. The file has to be named Microsoft.PowerShell_Profile.ps1 and has to be placed under $home\Documents\PowerShell\
When placed here it will be a user specific profile, you can also do system-wide profiles, take a look at $pshome.

The profile file I have created looks like this:

using namespace System.Management.Automation

# Set aliases

Set-Alias -name vim $home\Vim\vim90\vim.exe
Set-Alias -name vi $home\Vim\vim90\vim.exe

# Obey ctrl-d

Set-PSReadlineKeyHandler -Key ctrl+d -Function ViExit

# Enable ssh tab completion
# Credits to backerman and contributers at github for creating the code

Register-ArgumentCompleter -CommandName ssh,scp,sftp -Native -ScriptBlock {
param($wordToComplete, $commandAst, $cursorPosition)

function Get-SSHHostList($sshConfigPath) {
    Get-Content -Path $sshConfigPath `
    | Select-String -Pattern '^Host ' `
    | ForEach-Object { $_ -replace 'Host ', '' } `
    | ForEach-Object { $_ -split ' ' } `
    | Sort-Object -Unique `
    | Select-String -Pattern '^.*[*!?].*$' -NotMatch
}

function Get-SSHConfigFileList ($sshConfigFilePath) {
    $sshConfigDir = Split-Path -Path $sshConfigFilePath -Resolve -Parent

    $sshConfigFilePaths = @()
    $sshConfigFilePaths += $sshConfigFilePath

    $pathsPatterns = @()
    Get-Content -Path $sshConfigFilePath `
    | Select-String -Pattern '^Include ' `
    | ForEach-Object { $_ -replace 'Include ', '' }  `
    | ForEach-Object { $_ -replace '~', $Env:USERPROFILE } `
    | ForEach-Object { $_ -replace '\$Env:USERPROFILE', $Env:USERPROFILE } `
    | ForEach-Object { $_ -replace '\$Env:HOMEPATH', $Env:USERPROFILE } `
    | ForEach-Object { 
    $sshConfigFilePaths += $(Get-ChildItem -Path $sshConfigDir\$_ -File -ErrorAction SilentlyContinue -Force).FullName `
    | ForEach-Object { Get-SSHConfigFileList $_ } 
    }

    if (($sshConfigFilePaths.Length -eq 1) -and ($sshConfigFilePaths.item(0) -eq $sshConfigFilePath) ) {
        return $sshConfigFilePath
    }

    return $sshConfigFilePaths | Sort-Object -Unique
}

$sshPath = "$Env:USERPROFILE\.ssh"
$hosts = Get-SSHConfigFileList "$sshPath\config" `
| ForEach-Object { Get-SSHHostList $_ } `

# For now just assume it's a hostname.
$textToComplete = $wordToComplete
$generateCompletionText = {
    param($x)
    $x
}
if ($wordToComplete -match "^(?<user>[-\w/\\]+)@(?<host>[-.\w]+)$") {
    $textToComplete = $Matches["host"]
    $generateCompletionText = {
        param($hostname)
        $Matches["user"] + "@" + $hostname
    }
}

$hosts `
| Where-Object { $_ -like "${textToComplete}*" } `
| ForEach-Object { [CompletionResult]::new((&$generateCompletionText($_)), $_, [CompletionResultType]::ParameterValue, $_) }}

Exit your terminal and lauch it again, hopefully you can be a bit more productive with the new “features”

Re-Volt, even easier with rvgl launcher

The RVGL launcher project has made it even easier to install and play the good old racing game ReVolt.

To play on ubuntu, there are some prereqs you need to have in place, and not all of them are mentioned on the official RVGL launcher. To help you, I have gathered everything in a one liner, just here:

[copy]sudo apt install python3-wxgtk4.0 python3-requests python3-packaging p7zip-full xdg-utils libsdl2-2.0-0 libsdl2-image-2.0-0 libopenal1 libenet7 libunistring2 libjpeg8 libpng16-16 libtiff5 libwebp7 libvorbisfile3 libflac8 libmpg123-0 [/copy]

If you get an error on libwebp7, please try libwebp6

When the above is installed, download, extract and launch RVGL from here: https://re-volt.gitlab.io/rvgl-launcher/

When lauched, you may get an error about a lock file, dont worry, it’s only the first time you run it. Press “Download and Install” in th bottom, and you are ready to play. Remember to explore additional options in the rvgl launcher, it’s packed with cool features šŸ™‚

Get ready, set, race!

Gnome NetworkManager and vpnc – use nm-connection-editor

Gnome NetworkManager is awsome, supporting almost all kinds of VPN solutions, and all you need to do is install a plugin. My main issue is, that my customers often have a pre-build package (for Windows) containing a VPN client including configuration, so they actually dont know how it’s configured.

I usaually figure it out, and then its a breeze to click the network icon, select VPN and activate the connection, this is how it should be on all OS’es.

Unfortunatly there are some minor issues, one of them beeing that if you install the network-manager-vpnc module, you will have the option to create a VPNC based connection, and if you type everything right the first time, it will work. But….

You will not be able to edit the connection, it’s simply not listed when you select VPN settings (Ubuntu 22.04). using nmcli will list the connection, and you will be able to modify it, but I found another soluition, that is much easier.

nm-connection-editor

It will open a small GUI, letting you create, modify or delete all connections.

I have created a bug report here:
https://bugs.launchpad.net/ubuntu/+source/network-manager-vpnc/+bug/1951313
It has been confirmed, bot not fixed. Hopefylle it will be fixed in later releases.

Another issue, that I discovered today, is that you are not allowed to add the connection when creating it šŸ™ See some discussion here:
https://askubuntu.com/questions/1403896/cant-add-cisco-compatible-vpn-vpnc-on-network-manager-ubuntu-22-04
This issue is also not present in nm-connecttion-editor.

The connections will be working perfectly from the build in NetworkManager application in gnome.

systemd.mount replacing autofs

I have been a very satisfied user of autofs for several years. Using a laptop in multiple location without reboot, could often cause annoying timeouts with hard mounted NFS shares.

So I found autofs, which did a great job, especially the ghost option is nice, making you able to browse the filesystem when it’s not mounted, and only seeing a small delay when the autofs mounts it for you at your wish.

I have now discovered that systemd has a mount option: https://www.freedesktop.org/software/systemd/man/systemd.mount.html
It can be used as systemd units, but also directly from /etc/fstab where I prefer to have my mount options.

To replicate the autofs functionality, add something like this to your mount options:

noauto,x-systemd.automount,x-systemd.idle-timeout=60,x-systemd.mount-timeout=30,_netdev

The above options will mount when you try to access the share, and it will unmount after 1 minute of idling, and the _netdev tells the system not to mount it before the network is ready.
More or less the same functionality as autofs, but no need to install additional software.


Working with yaml files in vim

when working with yaml files, indentation is crucial. You can setup vim to make it a little easier, like this:

:set ai ts=2 sw=2 expandtab

The above will set tabstop and shiftwidth to 2 spaces, and make the tab key create spaces instead of tabs. So with this config, you will type 2 spaces, whenever you press tab, just like yaml likes it.

This can of course be put in your vimrc file.

zfs filling up the /boot device with snapshots

So, you are trying to update your system and get this:

Requesting to save current system state
ERROR couldn’t save system state: Minimum free space to take a snapshot and preserve ZFS performance is 20%.
Free space on pool “bpool” is xx%

The zfs filesystem is creating a snapshot every time you install a new kernel, this will add up, and suddenly you are not able to update anymore, and you are seeing a lot of error caused by this. If you are unlucky and reboot, it might even brake your system!

Running Ubuntu with zfs will create a pool called bpool as your /boot device. Continuing I will refer to this as bpool.


To get an overview of your zfs pools, use this command: zfspool list

kasper@AsusPro:~$ zpool list
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
bpool 1.88G 1.64G 238M – – 32% 87% 1.00x ONLINE –
rpool 114G 26.3G 87.7G – – 7% 23% 1.00x ONLINE

As you see, my bpool pool is 87% filled up, I have a problem.

To get an overview of snapshots in the bpool pool, use this command: zfs list -t snapshot | grep bpool

kasper@AsusPro:~$ zfs list -t snapshot | grep bpool
bpool/BOOT/ubuntu_nijvpt@autozsys_646r7v 8K – 92.8M –
bpool/BOOT/ubuntu_nijvpt@autozsys_5z8j40 8K – 92.8M –
bpool/BOOT/ubuntu_nijvpt@autozsys_rknbhc 0B – 92.8M –
bpool/BOOT/ubuntu_nijvpt@autozsys_cd3k48 0B – 92.8M –
bpool/BOOT/ubuntu_nijvpt@autozsys_e3s23t 0B – 185M –
bpool/BOOT/ubuntu_nijvpt@autozsys_teumkd 0B – 185M –
bpool/BOOT/ubuntu_nijvpt@autozsys_iaxh6k 88K – 279M –
bpool/BOOT/ubuntu_nijvpt@autozsys_8uh3k1 88K – 372M –
bpool/BOOT/ubuntu_nijvpt@autozsys_kpxpbq 88K – 187M –
bpool/BOOT/ubuntu_nijvpt@autozsys_3lafml 0B – 280M –
bpool/BOOT/ubuntu_nijvpt@autozsys_0gvki1 0B – 280M –
bpool/BOOT/ubuntu_nijvpt@autozsys_8j52ch 82.3M – 187M –
bpool/BOOT/ubuntu_nijvpt@autozsys_2sibtv 72K – 284M –
bpool/BOOT/ubuntu_nijvpt@autozsys_3t44c8 56K – 284M –
bpool/BOOT/ubuntu_nijvpt@autozsys_u58usc 0B – 284M –
bpool/BOOT/ubuntu_nijvpt@autozsys_3okd08 0B – 284M –
bpool/BOOT/ubuntu_nijvpt@autozsys_v18vua 0B – 284M –
bpool/BOOT/ubuntu_nijvpt@autozsys_f5185b 0B – 284M –
bpool/BOOT/ubuntu_nijvpt@autozsys_12i3wc 0B – 288M –
bpool/BOOT/ubuntu_nijvpt@autozsys_o6sgfl 0B – 288M –

Seems like zfs is not doing any housecleaning by itself, we need to help!
To remove a snapshot, we need to destroy it. This is done with the destroy option, like this:

sudo zfs destroy /path/to/snapshot

To remove all snashots in the bpool, you can use the following:

for i in $(zfs list -t snapshot | grep bpool | cut -d ” ” -f 1);do sudo zfs destroy $i; done

This will destroy all snapshots and free up your bpool pool.

If you don’t want to remove all snapshots, pipe your snapshots listing into tail or head. You can add a creation column to the list like this:

zfs list -t snapshot -o name,creation -s creation

This will list your snapshots in order of creation date. Use wc -l to count them and use head to get x amount fewer.

zfs list -t snapshot -o name,creation -s creation | wc -l
20

for i in $(zfs list -t snapshot | grep bpool | cut -d ” ” -f 1 | head -15);do sudo zfs destroy $i; done

I did try to format the commands, but as usual Word Press make things so complicated to use, that i had to revert all the special formatting, instead of spending the entire day figuring out why html tags are not respected in “enlighter inline code”

Scan for new drives on Linux – the easy way

Adding storage to servers is part of every day work. this can be anything from a virtual disk on a virtual machine, to LUN’s presented from some external storage system.

When the new disk has been added, ususally nothing happens. You need to scan for the new disk/disks in order to operate them. Theres a tool called rescan-scsi-bus.sh for doing this specific task, but sometimes you don’t have that, then you need to do it manually, and it’s actually not that difficult.

You then need to echo “- – -” to the scsi system. I have found a lot of different methods to do the same thing, a lot of them telling you to do something like: echo “- – -” >> /sys/class/scsi_host/hostxx/scan where xx can be anything, and sometimes there’s a lot, so typing all the different host numbers is not optimal. Then someone suggests a for loop, like: for i in $(/sys/class/scsi_host/hosts); do echo “- – -” /sys/class/scsi_host/host$i/scan; done

I have tried them all (I think) but in my opinion, the best and easiest solution is the one below, as it is pretty easy to remember:

echo "- - -" | sudo tee /sys/class/scsi_host/host*/scan

List your newly installed disk with lsblk and partition them with fdisk

Find and unmount CD drives with PowerCLI

Sometimes someone forgets to unmount the CD drive of a VM. This will interfere with operation tasks like setting a host in maintenance mode or using Update Manager.

I recently had this issue, and the “update manager compliance check” told me that several VM’s where connected to a CD drive, which may interrupt the update process. I quickly found my favorite tool for vcenter management, the powerCLI. The quick and dirty oneliner ended up like this:

get-cluster %clustername% | get-vm | where {(($_ | Get-CDDrive).isopath) -gt ""} | %{$_ | Get-CDDrive | Set-CDDrive -NoMedia -Confirm:$false}

Replace %clustername% with your cluster name, and all your troubles will be over šŸ™‚

I have always loved oneliners. I have spent the part time of my time working in bash terminals, where the history has always been persistent. Luckily Microsoft has also seen the light, and history is now persistent (and searchable) in powershell. Search for a phrase using ctrl+r, search further back by pressing ctrl+r again. I’m sure you will learn to love it, if you don’t already do.

Move the docker data directory

The default location for the docker data directory, when using overlay2 storage driver is /var/lib/docker. Maybe you want to move it to another location, and it’s actually pretty easy.

  1. Stop the docker service:
    sudo systemctl stop docker.service
  2. Copy the docker folder to the new location:
    sudo cp -rp /var/lib/docker /path/to/new/location
    The p in -rp will preserve ownership, permissions and timestamps
  3. Tell the docker daemon about the new location by editing the /etc/docker/daemon.json file with the following:
{
    "data-root": "/path/to/new/location"
}

4. Start the docker service:
sudo systemctl start docker.service

Thats it, the docker directory has been moved.

The above guide is tested on Ubuntu 20.04, but will probably be very much alike on other flavours.

Windows 10 missing ssh-copy-id

Microsoft is finally shipping Windows with SSH. On my Windows 10 machine, it’s OpenSSH:

PS C:\Users\Kasper> ssh -V
OpenSSH_for_Windows_7.7p1, LibreSSL 2.6.5

Somehow they missed to get the ssh-copy-id tool implemented, so we need to find another way of copying our ssh keys. Thankfully it’s not that complicated, we can use powershell (probably even cmd)

cat .\.ssh\id_rsa.pub | ssh -l username ssh_server "cat >> .ssh/authorized_keys"

This will concatenate (cat) your ssh key to the authorized_keys file on your ssh_server.

I’m using cat, to make it easy for Linux/unix admins, type, and get-content will also do, as both cat and type are aliases for get-content. I assume “type” will be available in cmd.exe