davidfuchs/davidfuchs-ca repository overview

⁠davidfuchs.ca

Source code for my personal website at davidfuchs.ca⁠.

⁠What is this?

A Hugo-based static site featuring:

• Blog posts about software development
• Curated lists of movies, music, and games (stored in YAML)
• About page

Uses the m10c theme⁠ for a clean, minimal design.

⁠Tech Stack

• Hugo static site generator
• m10c theme
• YAML for data (movies/music/games)
• Markdown for content
• Custom layouts and styling

⁠Music list

The music page (/music/) is a ranked list of songs, rendered by src/layouts/page/music.html from src/data/music.yaml.

⁠Data format

src/data/music.yaml has a single songs: key holding a list. Rank is the order of the list - there is no rank field, so reordering the entries reorders the page. Each song looks like this:

- song_name: "Cirice"
  artist: "Ghost"
  featured_artists: []          # list, may be empty
  album: "Meliora (Deluxe Edition)"
  album_artist: "Ghost"
  release_date: "2015-05-31"    # ORIGINAL release date of the song (see below); may be "YYYY" or "YYYY-MM"
  genre: "Metal"                # one of the 12 genres below; used for the page filter
  duration_sec: 362
  version: original             # original | remaster | live | acoustic | remix | edit | demo | instrumental | cover | other
  explicit: false
  isrc: null                    # a string like "USUM71234567" when a match was found
  apple_music_id: "1440946621"  # iTunes/Apple Music track id (kept as a string)
  apple_music_url: "https://music.apple.com/ca/album/cirice/1440946435?i=1440946621"
  artwork_url: "https://is1-ssl.mzstatic.com/image/thumb/Music114/v4/0d/be/08/0dbe08cb-a9bc-1783-b346-86eb51648f43/00888072015258.rgb.jpg/300x300bb.jpg"

Notes:

• apple_music_id is the key for everything else: it builds the "Apple Music" link and is how previews are fetched. isrc identifies the recording across other services (Spotify, Deezer, etc.) if we ever need to match them.
• Original albums. Where the catalog has it, a song points at the album it first came out on, not a "Greatest Hits" or "Essential" compilation. The exceptions are songs that were first released on a compilation or soundtrack (e.g. "September"), and a few that Apple's Canadian catalog does not have.
• release_date is the original release. It is the earlier of Apple's date and MusicBrainz's first-release date for the recording, so a 1983 song that was later put on a 1991 album shows 1983. MusicBrainz sometimes only knows a year or month, which is why some dates are "1983" or "1983-05".
• artist is the first-credited act; anyone else on the track is in featured_artists. Real bands whose names contain "&" or "and" (Simon & Garfunkel, Earth, Wind & Fire, ...) are kept whole. For classical pieces the ensemble or lead performer is the artist and the conductor and soloists are featured.
• version is inferred from the title and album text, so it is a best guess. "Remastered ..." is stripped from song_name because the version already says so; "(Radio Edit)" and similar are kept in the title.
• genre is one of: Metal, Hard Rock, Rock, Alternative, Pop, R&B/Soul, Hip-Hop, Country, Electronic/Dance, Classical, Soundtrack, Folk. They are mapped from Apple's 38 genres, an artist you tagged by hand in the old music.yaml keeps that tag, and an artist with several songs gets the majority genre so their songs filter together.
• A handful of songs are not in the Canadian storefront. Those were taken from another storefront (e.g. "The Time of No Time" uses a /gb/ link).
• isrc is null on about 50 songs where neither Deezer nor MusicBrainz had a recording of the same length (mostly radio edits, soundtrack and Ghost tracks).
• Preview URLs are deliberately not stored in the YAML (see below).

⁠Previews and the page

• Previews are 30-second clips from Apple's public iTunes Lookup API. At build time the template looks up every apple_music_id in batches of 150 (resources.GetRemote, cached by Hugo), builds an id -> previewUrl map, and renders a play button for each song that has one. IDs missing from the Canadian storefront are retried against the UK one. If a lookup fails the page still builds, just without those buttons (a warning is printed). Preview URLs live on Apple's CDN and can change, which is why they are resolved at build time instead of being stored.
• Playback uses one shared <audio> element that is only given a source when a button is pressed, so nothing is downloaded on page load, and starting a preview stops the previous one.
• Performance: artwork is lazy-loaded (loading="lazy", 100px thumbnails derived from the stored 300px URL), and rows use CSS content-visibility: auto so off-screen rows are skipped. The page is fine at under about 1,500 songs; far beyond ~5,000 it would be worth paginating.
• Top 500 only: the page renders just the first 500 songs in music.yaml (first 500 at the top of music.html). The rest stay in the file but generate no HTML, genre chip counts or preview lookups, so moving a song above or below #500 adds or removes it from the page.
• Filtering: the genre chips set ?genre=<slug>; a small script hides non-matching rows. Without JavaScript the full ranked list still shows and the Apple Music links still work.
• Styles live in src/assets/css/_extra.scss (.song-* classes).

⁠How the data was built

The list was generated once from my Apple Music library playlists; it is not regenerated by the build:

1. Track lists were read from each library playlist in the Apple Music web player (the page links each song to its catalog id).
2. Metadata (name, artist, album, release date, genre, duration, artwork, URL) came from the iTunes Lookup/Search API. Library tracks without a catalog link were matched by name, artist, album and duration.
3. ISRCs came from Deezer's public search API (matched on title, artist and duration within 3 seconds), since the iTunes API does not return them. A second pass (fill_isrc.py) added MusicBrainz and looser title matching.
4. The per-playlist files (now in tools/music_scratch/playlists/) were merged into music.yaml, de-duplicating by apple_music_id and ISRC (the first occurrence wins), with Favourite Songs first.
5. Cleanup passes (each is a script in tools/music_scratch/): songs on compilations were re-pointed at their original albums (repoint_albums.py), titles/collaborations/versions/genres were normalised (tidy.py), and release dates were replaced with the original release date from MusicBrainz (original_dates.py).

⁠Re-syncing from Apple Music

The tooling for the steps above lives in tools/music_scratch/. Like tools/list-editor, it is outside src/, so it is never part of the Hugo build or the Docker image (Hugo would also load anything under src/data/ as site data, which is why it does not live there).

tools/music_scratch/
  scrape.js     console snippet that reads a library playlist page
  raw/          scraped output, one <playlist>.json per playlist
  build.py      raw/ -> playlists/*.yaml via the iTunes Lookup/Search API
  enrich.py     fills in ISRCs from Deezer
  fill_isrc.py  second pass for songs still missing one (Deezer + MusicBrainz)
  repoint_albums.py  moves compilation tracks to the original album (iTunes)
  tidy.py       normalises titles, collaborations, versions and genres
  original_dates.py  original release dates from MusicBrainz
  playlists/    per-playlist YAML in playlist order, plus unmatched.yaml

music.yaml is now the source of truth: it has been cleaned up by the scripts above, had a few songs added from other storefronts, and is the file you will be ranking by hand. playlists/ is the pre-cleanup snapshot, so do not merge it over music.yaml wholesale. To pick up new music:

1. Sign in to the Apple Music web player, open the playlist, run scrape.js and save the result as raw/<playlist-slug>.json (a new playlist just needs a new file; its id is linked from the library sidebar).
2. cd tools/music_scratch && python3 build.py regenerates playlists/ and unmatched.yaml, then python3 enrich.py fills ISRCs. Both need network access and only use the Python standard library. API responses are cached in cache.json / isrc_cache.json (git-ignored), so reruns are quick.
3. Compare playlists/ against src/data/music.yaml by apple_music_id (or isrc), copy over the entries that are new, and place them where you want them in the ranking. Then run the cleanup scripts so the new entries follow the same conventions: python3 repoint_albums.py, review repoint_candidates.json and apply with --apply, then python3 tidy.py --apply, then python3 original_dates.py followed by --apply. The scripts only touch entries that still need work, so they are safe to re-run. Check unmatched.yaml for songs the Canadian storefront could not match, and look for them under another storefront (the track lists from an artist lookup, lookup?id=<artistId>&entity=song, find things the name search misses).

⁠Known issues

• tools/list-editor still expects the old schema (artists: with a genres: array) and cannot edit the new songs list yet. Reorder music.yaml by hand until it is updated.