Rubenerd: Erroneous systemd Ghost errors in FreeBSD

Too long, didn’t read? I reinstalled Ghost and copied across the content and configuration file. It worked.

I get a bit into the weeds here, and there isn’t much of a satisfying conclusion, but I’m documenting the process in the hopes that parts of it might help someone. A few Ghost blogs I run for people on a FreeBSD virtual server failed last night, and were only serving their content via local caches.

I tried restarting one of them in its FreeBSD jail:

$ ghost start example-blog
	
Process manager 'systemd' will not run on this system, 
	defaulting to 'local'
[..]

That’s true, FreeBSD doesn’t use systemd. But then I got systemd-related errors anyway, which made no sense:

✖ Checking systemd unit file
✖ Checking systemd node version
One or more errors occurred.
	
1) SystemError
	
Message: Unable to load or parse systemd unit file
	
2) SystemError
	
Message: Unable to determine node version in use by systemd
Help: Ensure 'ExecStart' exists in 
	/lib/systemd/system/ghost_ghost-local.service 
	and uses a valid Node version
	
Debug Information:
	OS: FreeBSD, v12.2
	Node Version: v14.17.0
	Ghost Version: 4.8.4
	Ghost-CLI Version: 1.17.3
	Environment: production
	Command: 'ghost start local'

Checking logs:

$ ghost log --error
Process manager 'systemd' will not run on this system,
	defaulting to 'local'
[2021-04-22 16:22:17] ERROR
NAME: InternalServerError
CODE: SHARP_INSTALLATION
MESSAGE: Sharp wasn't installed

Okay, let’s install it then:

$ npm install sharp

Bingo, the most recent version requires an external library on BSD and Solaris:

npm ERR! code 1
npm ERR! path /var/ghost/node_modules/sharp
npm ERR! command failed
[..]
npm ERR! sharp: Please see 
	https://sharp.pixelplumbing.com/install for required dependencies
npm ERR! sharp: Installation error: 
	BSD/SunOS systems require manual installation of libvips >= 8.10.6

I searched the ports system and found it, courtesy of danilo@. I installed:

$ sudo pkg search vips
==> vips-8.10.6_1  Free image processing system
	
# sudo pkg install -y graphics/vips
[..]
Number of packages to be installed: 101
The process will require 768 MiB more space.
140 MiB to be downloaded.

Holy dependencies Batman. Good thing I’m running this in a FreeBSD jail so I’m keeping this all local to the ghost install, and not clobbering system versions of things.

Now I could try installing sharp again:

$ npm install sharp
added 32 packages, changed 1 package, and audited 158 packages in 6s

That looked promising. Alas, the same Ghost error from the start of this post.

I tried rebuilding in the Ghost directory:

$ npm rebuild
npm ERR! code 1
npm ERR! path /var/ghost/node_modules/sharp
npm ERR! command failed
npm ERR! command sh -c (node install/libvips && node install/dll-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy)
npm ERR! sharp: Detected globally-installed libvips v8.10.6
npm ERR! sharp: Building from source via node-gyp
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using node-gyp@3.8.0
npm ERR! gyp info using node@14.17.0 | freebsd | x64
npm ERR! gyp ERR! configure error 
npm ERR! gyp ERR! stack Error: Command failed: /usr/local/bin/python3.8 -c import sys; print "%s.%s.%s" % sys.version_info[:3];
npm ERR! gyp ERR! stack   File "<string>", line 1
npm ERR! gyp ERR! stack     import sys; print "%s.%s.%s" % sys.version_info[:3];
npm ERR! gyp ERR! stack                       ^
npm ERR! gyp ERR! stack SyntaxError: invalid syntax

This forum post suggested removing the Sharp directory, then try building again:

$ cd /var/ghost/node_modules
$ mv sharp sharp-backup
$ npm install sharp

Same error as above.

At this stage I just wanted to get back to my Sunday breakfast. I mulled rolling back to yesterday’s automatic ZFS snapshot, but in the end I installed a new version of Ghost, and copied across the config:

$ mv /var/ghost /var/ghost-borked
$ mkdir /var/ghost
$ cd /var/ghost
$ ghost install
$ rm -rf /var/ghost/content
$ cp /var/ghost-borked/config.production.json /var/ghost/
$ cp -R /var/ghost-borked/content /var/ghost/content

It worked. Either npm or Ruben borked a bunch of dependencies. I could get back to my breakfast, and a partner in crime could get back to her writing.

Things I learned today:

OpenZFS is a lifesaver. Had stuff really gone belly-up, I could have rolled back to an earlier snapshot and troubleshooted on my own time.
The fact you can rebuild an entire Ghost install in a few commands, and import the config and content from another install is awesome.
Ghost is way faster and easier for end users, but its dependency tree is so much more brittle than WordPress. I’ve been running the latter for people since 2005 without equivalent errors, save for that one adventure with PHP’s MySQL package in 2010-ish.
I need to skill up on node and npm package management more than I thought.

Author bio and support