Archive: 2023

One of the reasons I host using HTMLy is that it is quite simple. No DB's to manage, just PHP and some .MD files.

I have been upgrading many of my servers that run on 18.04 and sometimes PHP doesnt make this as easy as it should be. My old instance ran on Apache. Nothing against Apache, I am just increasingly more familiar with NGINX. So a do-release-upgrade failed, and rather than take the time to properly fix the post-upgrade issues, I just rolled a new VPS (and saved some cash) and moved my files.

Process was fairly simple though.

1. Build new server, install and configure SSH, firewall, user etc.
2. Install php-fpm and nginx
3. Configure nginx
4. Copy the root webserver files from the old source to the new.
5. setup my rsync accounts and re-configure my backups

This is all pretty boilerplate stuff. I use UFW for the firewall and simply limit SSH connections to my known IP's I tend to work from. I configure SSH with normal settings, like limiting access to groups, only using key based auth etc.

PHP-FPM needs nothing special, though NGINX did need some tweaking. The recommended config, at least for Debian/Ubuntu does work out of the box. I am not using PHP-cgi or TCP sockets but rather the default Unix sockets.

Here is the sample config that does work.

server {
    listen 80 ;
    listen [::]:80;

    server_name wallyswiki.com www.wallyswiki.com;
    root /var/www/html;

    # Add index.php to the list if you are using PHP
    index index.php index.html index.htm index.nginx-debian.html;

    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log error;

    location / {
            try_files $uri $uri/ /index.php?$args;
    }

    # Commented out as it broke the admin pages
    #location ~ /config/ {
    #    deny all;
    #}

    # pass PHP scripts to FastCGI server
    #
    location ~ .php$ {
            include snippets/fastcgi-php.conf;
    #
    #       # With php-fpm (or other unix sockets):
            fastcgi_pass unix:/run/php/php8.1-fpm.sock;
    #       # With php-cgi (or other tcp sockets):
    #       fastcgi_pass 127.0.0.1:9000;
    #       fastcgi_index  index.php;
            fastcgi_param  SCRIPT_FILENAME
            $document_root$fastcgi_script_name;
            include        fastcgi_params;
    }

    # deny access to .htaccess files, if Apache's document root
    # concurs with nginx's one
    #
    location ~ /.ht {
            deny all;
     }
    }

After this its simply setting up certbot/Lets Encrypt.

Finally, since there is no DB or anything to work off of. You can simply copy over your files and make sure the permissions (namely the www-data:www-data users and groups) are set properly.

For me, I simply created a tar file, used SCP to copy the file over and extracted it. Though you could easily do this with rsync as well, and this would be especially useful if you are trying to keep multiple locations in sync.

UPDATE

I noticed RSS wasnt working. Nginx was throwing the following in error.log

2023/05/12 00:33:36 [error] 16320#16320: *1090 FastCGI sent in stderr: "PHP message: PHP Fatal error:  Uncaught Error: Class "SimpleXMLElement" not found in /var/www/html/system/vendor/suin/php-rss-writer/src/Suin/RSSWriter/SimpleXMLElement.php:9

Therefore I simply installed php8.1-xml and issue is fixed.

sudo apt install php8.1-xml
sudo service php8.1-fpm restart
sudo service nginx restart

So you might have tried upgrading your Unifi devices via the controller and for some reason it did not work. Usually this means that the download of the firmware-file failed and chances are big this happens for one of your Switches. In my usecase I am manually upgrading the US-8-60W.

First you download the latest .bin-file from the Downloads page. Make sure to put it in the Downloads-folder on your Macbook in order to follow the steps I have specified below, else you need aadjust accordingly.

Open the Terminal app and copy the firmware over

scp ./US.bcm5334x_5.43.23+12533.201223.0319.bin ubnt@10.0.1.2:/tmp/fwupdate.bin

If this is the first time establishing an ssh connection to the device it will ask you if you want to continue connecting. Type ‘yes’ and press Enter.

Add the password.

If you don’t remember the username/password to ssh into your devices you can check them in your controller. Go to Settings > System Settings > Controller Configuration > Device SSH Authentication. If you did not set it up yet you can do so now.

The Terminal will show you the progress of uploading the .bin-file to /tmp/fwupdate.bin which for me took about 20 seconds.

After the file is uploaded you now need to ssh into the device and execute the upgrade command:

ssh ubnt@10.0.1.2

Add password. (this is the same user/password you used before to upload the file). Then:

syswrapper.sh upgrade2 &

You will see the progress again and in total just give it 5 minutes and reload the controller to verify the upgraded firmware version.

That’s it.

Source: https://www.carlobloks.com/blog/manually-upgrading-unifi-firmware/

With MacOS certain simple things can be quite.....perplexing. Like 3 finger middle click on a touchpad, you need an App for that.. Such is the way with many of the seemingly "default" options in MacOS and probably the reason so much is "appified" in that ecosystem. The Apple devs probably just dont worry about such "minutia" that has made its way into other OS system defaults because they can just rely on a swift developer to make an app and try and charge $0.99 in the AppStore to enable something that would otherwise be default. So simple things like keyboard compatibility, or how responsive you want to dock to be, or things like snapping windows to a side of the screen need these weird customization settings. Thats enough ranting through, it is what it is in that ecosystem and Apple does some things pretty well. Their security defaults are solid, especially if you arent someone that is prone to caring much about things like full disk encryption because you simply dont understand it.

Anyhow, not the topic for today. The MacOS terminal isnt bad, comparable to most of the others I have used. But one thing that nagged me about the terminal app is how the Home/End keys basically don't behave.

Heres a quick fix.

Pull up the Terminal Settings and navigate to your default profile and go to the keyboard options.

Fill it out like so and hit OK:

Here are some other *nix friendly keycodes:

• home :: \033OH
• end :: \033OF
• F1 :: \033[11~
• F2 :: \033[12~
• F3 :: \033[13~
• F4 :: \033[14~

Source is from an old blog post that seems to have been hijacked to advertise spam. But here is the archive of the post.

So I had a weird issue the other day, just after I had created an extent and added it to a volume group to extend my plex disk (mostly for more scratch space for the transcoder to do its job).

Anyway, I got an alert in Zabbix that the disk was nearly full.

This was odd, since I had just increased the disk some 30+ GB's. A quick look at the server and sure enough df showed that it was in fact used

df -h

It looked something similar to this.

Notice how the main root mount point was showing high usage

Note: I fixed this before i wrote this post. So I artificially re-created the condition for the screenshots.

I have a small script that I often use. I call it treesize, but in reality its just du with some extra stuff and I often use it to find where storage is being eaten up.

Heres the script:

#/bin/sh
du -k --max-depth=1 | sort -nr | awk '
     BEGIN {
        split("KB,MB,GB,TB", Units, ",");
     }
     {
        u = 1;
        while ($1 >= 1024) {
           $1 = $1 / 1024;
           u += 1
        }
        $1 = sprintf("%.1f %s", $1, Units[u]);
        print $0;
     }
    '

And here is what an output would look like.

Now its worth noting a few things at this point.

1. Plex uses cifs mounts setup via fstab for its actual library content.
2. The NAS that stores those mounts is backed up nightly.
3. Plex itself is a Virtual Machine and also backed up nightly.

As a result I will sometimes have plex oddly lose its cifs mounts.This is likely due to the VM being "stunned" during backup, but could also happen during the btrfs snapshot and backup done on the NAS. Frankly, I never had the time nor the inclination to look too far into it because simply updating the server nightly and rebooting it fixes the issue in 95% of the cases.

So i got to googling (well "duckduckgoing") and came across some of the standard advice such as check to make sure inodes arent exhausted or run lsof to make sure a deleted file wasnt held up by a running process. But then i got to this page about bind mounts and finally this one that really got me thinking.

I do have plex setup to record some over-the-air TV (OTA) stuff, mostly local news and kids shots. What would happen if plex lost its cifs shares inexplicably AND Plex did a ton of recording to that same path location before I had time to catch it?

So I stopped the plex service and unmounted all shares, then re-ran my du/treesize script.

Sure enough, some extra storage utilization appears. And it all appears in the very share that I used Plex to record over the air TV to.

Note: As stated, in this example I used fallocate to create some large files to use as examples. My actual directory looked something like this snap which I grabbed from backups.

Sure enough that's exactly what happened. So in this case, its worth looking at any mounts you may be using and making sure that you don't have some files inexplicably loaded up in folder locations you typically use to mount shares...

Or just rebuild the server, I am not your parent.