Unless they are explicitly bound to loopback, which I thought was the
default, but it is not. likely related to the interface between
bridges and ip tables in the Linux kernel, but anyhow, get literal
about the portmap interface address to prevent exposing the database
to the entire internet. With thanks to the friendly heads up email
from the German Federal Republic via Hetzner.
stream-harmony.lisp:
- scan-music-library-files: use native-namestring instead of namestring
when collecting file paths, producing real OS paths without SBCL's
backslash escaping of brackets (e.g. [FLAC] not \[FLAC])
- Use parse-native-namestring for root directory entry point so dirs
with brackets are not treated as wildcard patterns
cl-streamer submodule updated to d57a268:
- play-file, read-audio-metadata, play-list retry: same native-namestring
fix to prevent double-escaping through parse-native-namestring
Fixes FLAC FILE slot unbound errors and SIGSEGV memory fault that
crashed the shuffle pipeline after consecutive failures on bracket paths.
Replace in-tree cl-streamer/ with git submodule pointing to
glenneth1/cl-streamer (https://github.com/glenneth1/cl-streamer).
ASDF discovers cl-streamer via source-registry :tree scan — no
config changes needed. Submodule tracks master branch.
Build verified: all cl-streamer systems load from submodule.
- New make-pipeline function: single declarative call creates server,
mounts, encoders, and pipeline wiring from an output spec
- Pipeline owns encoder lifecycle (pipeline-encoders slot, auto-cleanup)
- Pipeline owns server when it creates one (pipeline-owns-server-p)
- Hook system wired: pipeline-add-hook fires on track-change and
playlist-change via pipeline-fire-hook
- stream-harmony.lisp slimmed: start is 1 make-pipeline + 2 hooks,
stop is 1 pipeline-stop call (cleanup automatic)
- Removed global encoder variables from Asteroid glue layer
- Backward-compatible: dj-session.lisp unchanged, cl-streamer:*server*
still set for legacy callers
Runtime verified: audio streams, metadata displays, crossfades work.
- asteroid/stats/current: add explicit :limit 120 :timeout 60 (was default 60/60s)
- now-playing, now-playing-inline, now-playing-json: change from :limit 10 :timeout 1
to :limit 30 :timeout 60 — the 1-second window was too aggressive and likely
triggering r-simple-rate's negative-amount corruption bug
These endpoints are polled every 5-30s by the player frame, admin dashboard, and
popout player. With multiple tabs/frames sharing a session, the old limits were
easily exceeded, producing 429 responses that cascaded into audio error events.
- Keep *is-reconnecting* true until 'playing' event fires, preventing
pause/stalled handlers from triggering new reconnect cycles mid-flight
- Add exponential backoff for stall retries (5s, 10s, 20s... max 60s)
- Give up auto-reconnect after 10 stall attempts, show manual retry
- Add *stall-count* tracking, reset on successful playback
- Add *user-paused* guard to muted-tab pause handler
- Increase play() delay from 200ms to 500ms after load() for reliability
Fixes: AbortError from play()/pause() race, 429 Too Many Requests from
aggressive reconnect hammering, infinite reconnect loop on muted tabs.
- Replace default float packer with int16 packer (mixed:make-packer :encoding :int16)
cl-mixed now handles float→s16 conversion in optimized C code instead of
per-sample Lisp loop. Halves pack buffer memory (2 vs 4 bytes/sample).
- Remove float-to-s16 helper (no longer needed)
- Fix resume-from-saved-state: when saved playlist differs from currently
scheduled playlist, use the scheduled one from the beginning instead of
continuing the old playlist. Prevents stale playlist playing after restart.
Setting (mixed:encoding pack) :int16 after server creation did not
change the pack's internal buffer format — data was still written as
float but read as int16, producing garbage audio.
Added TODO comment to investigate correct API for setting pack encoding
at creation time. The float→s16 conversion in Lisp works correctly.
- Fix api-post: skip Content-Type header on empty-body POSTs (Hunchentoot 400)
- Fix api-post/api-get: unwrap Radiance data wrapper, add try/catch + console logging
- Fix search-library-tracks: use raw SQL with parameterized ILIKE (S-SQL :offset broken)
- Fix search-library-tracks: quote file-path column name for Radiance hyphenated columns
- Add pipeline-stop-all-voices: immediately silence all Harmony voices on mixer
- Fix pause-auto-playlist: clear queue + skip + stop all voices (no more overlap)
- Override get-now-playing-stats during DJ session to show active deck info
Comprehensive design doc for the live DJ mixing feature:
- Dual-deck library mixing with crossfader (constant-power curve)
- External audio input: local sound card (ALSA/Pulse/JACK) and
network audio (Icecast source protocol for remote DJs)
- Session lifecycle with auto-playlist pause/resume and watchdog
- API endpoints, backend classes, frontend layout
- Phased implementation plan
- Open questions for team discussion
- Add *resumed-from-saved-state* flag to prevent scheduler's db:connected
trigger from overwriting resumed playlist position with full playlist
- Use sb-ext:parse-native-namestring in play-file to prevent SBCL from
interpreting brackets in directory names (e.g. [WEB FLAC]) as wildcard
patterns, which caused non-simple-string pathname components that broke
cl-flac's CFFI calls
- cl-cron uses local time, not UTC: add utc-hour-to-local-hour
conversion so schedule hours fire at correct UTC times
(e.g. 12:00 UTC now fires at 15:00 local on UTC+3)
- Wrap each taglib field read in safe-tag with per-field error
handling so a type error in one field (e.g. album with non-simple
string) doesn't crash play-file or skip the track
- Use (coerce ... 'simple-string) instead of copy-seq for
guaranteed simple-string output from ensure-simple-string
- Prevent play-list thread death on scheduler playlist change:
drain-queue-into-remaining drains full scheduler queue at once,
updates loop-queue reference so repeat replays correct playlist,
top-level handler-case prevents thread from dying silently
- Fix taglib SIMPLE-ARRAY CHARACTER type errors:
ensure-simple-string coerces metadata strings and trims whitespace
- Fix FLAC::FILE slot unbound errors:
retry once with 200ms delay for transient init failures
- Improve playback state persistence:
save playlist path alongside track file so restart loads the
correct playlist instead of always falling back to stream-queue.m3u
- Startup now uses resume-from-saved-state to resolve saved playlist
and track position, falls back to stream-queue.m3u only if no state
- Enable loop-queue in play-list: repeats current playlist when tracks run out
- next-entry checks queue first so scheduler-queued playlists override repeat cycle
- Prevents silent stream / client reconnect loops between scheduled playlist changes
- Delay metadata/track-change notification by 1s after crossfade completes
- Log 'Loading next:' instead of 'Now playing:' during crossfade prep
- Add diagnostic logging: track duration check, crossfade trigger time
- harmony-load-playlist defaults to skip=nil: scheduler queues tracks
without interrupting current playback
- Save current track to .playback-state.lisp on each track change,
resume from saved position on restart
- Replace ~50 format-t debug statements in auth with log:info/log:warn
- Remove password hash logging for security
- Add .playback-state.lisp to .gitignore
- Use total listener count across all mounts instead of per-mount
(asteroid.lisp icecast-status API, stream-harmony.lisp now-playing)
- Fix recently-played.lisp: equal -> = for ParenScript string comparison
- Remove non-existent asteroid-shuffle.mp3 mount from recently-played JS
- Map all mount references to existing asteroid.mp3/asteroid.aac
Server-side fixes (stream-server.lisp):
- Add CORS preflight (OPTIONS) request handler for browser crossorigin audio
- AAC clients start from current buffer position instead of burst to avoid
ADTS frame alignment issues that caused browser decode errors
- Upgrade client stream error logging from debug to warn for diagnostics
- Add send-cors-preflight function with proper Access-Control headers
Frontend fixes (stream-player.lisp):
- Rewrite reconnect-stream to reuse existing audio element instead of
creating a new one, preserving browser user gesture context and preventing
NotAllowedError on autoplay after reconnect
- Unify stream config: both curated and shuffle channels use same mount
points (asteroid.mp3/asteroid.aac) since cl-streamer has a single pipeline
- Remove non-existent /asteroid-shuffle.mp3 mount reference that caused
404s and broken pipe cascade when switching to shuffle channel
- Map :low quality to same MP3 mount (asteroid-low.mp3 not yet available)
Note: Channel selector preserved for future multi-stream support.
Recently-played API works correctly; frontend rendering to investigate separately.
CORS fix (icy-protocol.lisp):
- Add Access-Control-Allow-Origin: * to stream response headers
- Browser audio player can now connect cross-origin (port 8080 -> 8000)
Auto-start (asteroid.lisp -main):
- Start cl-streamer pipeline automatically on boot
- Load stream-queue.m3u and begin playback immediately
- Wrapped in handler-case so streaming failure doesn't block web server
Mount names (stream-harmony.lisp):
- Renamed /stream.mp3 -> /asteroid.mp3, /stream.aac -> /asteroid.aac
- Matches existing frontend URLs, zero template changes needed
Icecast bypass (asteroid.lisp, listener-stats.lisp):
- Front page uses get-now-playing-stats instead of icecast-now-playing
- check-icecast-status returns cl-streamer status when pipeline is active
- check-liquidsoap-status returns N/A when using cl-streamer
- asteroid/icecast-status API returns cl-streamer data directly
- poll-and-store-stats uses cl-streamer listener counts directly
- Eliminates hanging HTTP requests to port 8000 for Icecast XML
Tested: full browser streaming working end-to-end
- Add taglib dependency to cl-streamer/harmony system
- Add read-audio-metadata: reads artist/title/album from FLAC/MP3 tags
- Add format-display-title: builds 'Artist - Title' from tags, falls back to filename
- Add update-all-mounts-metadata: updates ICY metadata on all mount points
- Defer metadata update during crossfade until fade completes (listeners hear correct track)
- Fix play-list wait loop: was nested inside crossfade conditional, first track never waited
- Remove filename-derived :title from test playlist (taglib reads real tags now)
- Fix FDK-AAC C shim: use proper OUT_BITSTREAM_DATA=3 constant and INT types
- Add frame-aligned PCM accumulation buffer to AAC encoder for clean output
- Add fdkaac_encode and fdkaac_close to C shim (all FDK-AAC calls in C)
- Implement crossfade between tracks (3s overlap, 2s fade-in/out)
- Tune buffer: 90% drain sleep for encoding headroom, 64KB burst-on-connect
- Add FFI bindings for new shim functions
- Rewrite README.org with full architecture docs and integration plan
- Broadcast buffer: single-producer multi-consumer ring buffer with
per-client read cursors. 32KB burst-on-connect for fast playback.
Never blocks producer (overwrites old data for slow clients).
- Sequential playlist: play-list runs tracks one at a time using
Harmony's on-end callback + condition variable for completion.
- ICY metadata: set-now-playing called on each track change.
- Fixed string vs pathname bug in harmony:play (etypecase mismatch).
- Debug logging for client disconnect diagnosis.
Verified: browser plays shuffled FLAC playlist via 128kbps MP3 stream.
Major changes:
- streaming-drain: Custom drain that captures PCM from Harmony's pack
buffer (raw IEEE 754 floats in unsigned-byte-8 array), converts to
signed-16 PCM via CFFI, encodes to MP3 via LAME, and writes to
stream server's ring buffer
- Fixed ring buffer deadlock: buffer-read/buffer-write held lock then
called buffer-available which tried to acquire same lock. Created
internal %buffer-available/%buffer-free-space without locking.
- Fixed ring buffer zero-length guard for unbound variable in finally
- Fixed sleep duration in drain: was dividing raw byte count by
samplerate, now correctly converts to frames first
- Added flexi-streams wrapper for bivalent HTTP socket I/O
- Exported all public API symbols from cl-streamer package
- Added test-stream.lisp end-to-end test script
Verified: Amon Tobin FLAC -> 128kbps MP3 stream at localhost:8000
file reports: MPEG ADTS, layer III, v1, 128kbps, 44.1kHz, JntStereo
- harmony-backend.lisp: Audio pipeline connecting Harmony to encoders
- Uses :dummy drain to capture PCM without audio output
- Reads from packer bip-buffer via request-read/finish-read API
- Encodes PCM to MP3 via LAME and feeds to stream server
All three systems compile cleanly:
cl-streamer, cl-streamer/encoder, cl-streamer/harmony
- Architecture document outlining Icecast/Liquidsoap replacement
- Core streaming server with ICY metadata protocol support
- Thread-safe ring buffer for audio data
- Mount point abstraction with metadata updates
- Multi-client connection handling
This is experimental groundwork for integrating with Harmony/cl-mixed
and playlisp/parsector for a pure CL streaming solution.
API-ENDPOINTS.org:
- Add documentation for all 90+ API endpoints
- Stream Queue, Scheduler, Track Requests, User Playlists
- Favorites, History, Profile, Account endpoints
- Frontend Partials, Utility endpoints
- Now covers 100% of define-api endpoints in codebase
PROJECT-OVERVIEW.org:
- Tidy up tables and formatting
Brian O'Reilly
Glenn Thompson
Luis Pereira