A python TUI and misc utilities for building and using a SWI-Prolog RDF store.

a lot to unpack here more or less what it do:

• “indexes” a digital music library (bunch of music files) as a set of linked data according to the schema rendered here and specified here. To understand the rendering
  • all boxes are URI references referenced in the schema.
  • cyan boxes are predicates connecting their domain to their range.
  • pink arrows denote subclassing.
  • dark orange boxes denote resources not defined in this schema but subclassed by resource classes in the schema.
  • light orange boxes are typeclasses defined in the XSD schema
• gives a terminal interface that browses resources in the database. Trees form the basis of this interface. an API on top of the prolog store accepts queries which can be rendered directly as trees in the interface. queries can be selected interactively and are easy to create without modifying the interface. the api closely mimics directly querying the prolog store interactively.
• uses mpd to play music and features an window in the terminal interface to control the mpd queue. basic commands to be documented but mpd_player.py is short and clean and can be integrated into any urwid interface (for use without an rdf database.) songs in the instance window of the the interface can currently be added to the mpd queue by artist, album, and individual song.
• each song play thru mpd can be added as an ‘xcat:accessed_during’ property automatically with a systemd daemon.

this is really a knowledge graph explorer built on SWI-Prolog’s Semantic Web library. It works with data structured using RDF and RDFS and relies on XSD Typed literals.

Note: this /quickstart/ is more appropriately thought of as a bootstrapping guide for recreating my setup with a new system and new data. at some point I might make a docker container or tarball that has a demo knowledge graph for checking out the application easily

Additional Note: this project is still somewhat in the /prototype/ stage of development and I haven’t put much thought into deploying it outside that context. these instructions should be accessible to a relatively new linux user with basic understanding of a terminal environment but will speed through stuff like linux audio, python virtual environments, and systemd services. idk the best places to start reading about these but they’re all worth learning about.

here are the minimal instructions for setting up on Debian Bullseye. Probably will work on other debian-based Linux distros like Ubuntu or Mint (idk, lmk?). Windows and OSX installation seems hypothetically possible? Designed to run on a modest linux server, a modern raspberry pi will probably do just fine.

Tested with these versions, mileage may vary for anything older (will definitely break on anything older than python 3.8)

Note: I’m devloping on SWI-Prolog 8.3.22 but also regularly using 8.2.4 which is the latest version currently accessible from the Debian Stable apt repository.

The following steps install needed dependencies. Non python programs are installed at the system level, python dependencies are installed to a virtual environment in the project directory. This is useful for a “development environment” where scripts and the interactive python shell can be used to interact with the RDF database. The downside for regular use is that the virtual environment must be sourced in the terminal we’re running in.

Important Note: Really don’t recommend installing the python dependencies at a system level. a growing number of packages are installed from pinned git commits instead of actual pip releases which seems likely to wreak dependency havoc on a system level python install.

sudo apt install mpd mpc swi-prolog # non python dependencies
git clone https://github.com/xeroxcat/doubletree.git
cd doubletree && python3 -m venv ./virtual-env
source ./virtual-env/bin/activate # enter the vitrual environment
python3 -m pip install -r requirements.txt # install python dependencies
#deactivate # exit the virtual environment (or don't - we'll need it later)

mpc is not really a dependency but can be useful for debugging mpd and toggling the play state without having the UI open. Its the standard command line mpd client.

I’ve always needed to make some changes to /etc/mpd.conf in order for it to play. Primarily enabling ALSA or PulseAudio depending on whatever I have set up. In general:

1. if I have pulse installed (pactl --version is a valid command) use pulse output as described in this question and this followup
2. otherwise I use the default alsa config by uncommenting it from /etc/mpd.conf.

add to /etc/mpd.conf:

audio_output {
       type     "pulse"
       name     "My PULSE Device"
       server   "127.0.0.1"
}

add to /etc/pulse/default.pa:

load-module module-native-protocol-tcp auth-ip-acl=127.0.0.1

this doesn’t set anything up but will make sure we can access and save the Prolog RDF store. Currently the doubletree/data/pl_store must be manually created for this.

# assume we're still in the root project folder
mkdir data/pl_store
cd src && ./init.pl # should run without errors
# exit the prolog shell with "halt."

(this definitely deserves some context)

I reccommend using the beets config file at data/config.yaml. pass it to any beets command listed below with the -c option (-c data/config.yaml) or create a symlink to it at ~/.config/beets/config.yaml. this config file adds the discogs and bandcamp source plugins and sets beets to not actually touch any files it pulls data for. Haven’t fully explored the implications of using beets this way but it prevents any mistakes made in the beets import process from overwriting info in the file metadata (which might be more correct).

Import all music into the beets library: fyi with ~200gb of rather poorly sorted music this took me about a day

beet import <path to your music dir>

If music isn’t centralized in a single directory, you can repeat this command with every directory you want to import.

# assume we're still in the src/ directory
./beets_to_rdf.py <path to your music dir> <path to other music dir> ...

probably deserves some explanation

src/mpd_monitor/ contains a script to monitor the mpd service and add listens to the rdf database. It also contains a service file that can be installed to systemd. I’ve only really tested installing as a user service so instructions for that are included here.

1. edit mpd_monitor.service so the paths point to the project directory and mpd_monitor.py.
2. edit mpd_monitor.py so the executable invoked in the hashbang points to the python executable in your virtual environment.
3. symlink, move, or copy mpd_monitor.service to ~/.config/systemd/user/mpd_monitor.service.
4. systemctl --user start mpd_monitor.service to start. tail the journal with journalctl --user -f to make sure it started correctly. replace start with enable to make it run on startup (good idea)

./doubletree.py

doubletree interface isn’t quite mature enough to be a schema editor, probably will be a while before it is mature enough to be a good schema editor. currently I use triple20, an RDF-editing GUI that is i think entirely programmed in Prolog and is at least 18 years old as of 2021 (converted to SVN from CVS in 2003). It is quirky but was the most usable tool that still seemed to run ok in 2021.

see Schema Renderer README for a brief description of the rendering tool; its clunky.

currently all in the src project dir

(run with -h flag for options) this script was made to update the paths and hashes for each direntry in the database. This was designed to let me update the paths to music in rdf without having to reimport it all into beets and then reimport that all in to a clean rdf db. It also will add any new files (as generic xcat:File resources) and directories.

To do this is rehashes each file and dir in the specified paths and checks for the hashes in the rdf db. any direntry resources not found are deleted.

similar to the above script, this script updates the paths in the rdf db but more naively (and quickly). it does a basic string replacement on all paths in the database. it is intended for when all files in the db under a common root path move to a new root path. does no checking for validity.

delete the entire db. probably not a good idea. execute permissions disabled for that reason. run it with sh clean_db.sh if needed. might be necessary to do this before running rdf_from_beets.py if a previous run had ended midway thru adding files to the rdf db.