Comprehensive design doc for the live DJ mixing feature:
- Dual-deck library mixing with crossfader (constant-power curve)
- External audio input: local sound card (ALSA/Pulse/JACK) and
network audio (Icecast source protocol for remote DJs)
- Session lifecycle with auto-playlist pause/resume and watchdog
- API endpoints, backend classes, frontend layout
- Phased implementation plan
- Open questions for team discussion
- Add *resumed-from-saved-state* flag to prevent scheduler's db:connected
trigger from overwriting resumed playlist position with full playlist
- Use sb-ext:parse-native-namestring in play-file to prevent SBCL from
interpreting brackets in directory names (e.g. [WEB FLAC]) as wildcard
patterns, which caused non-simple-string pathname components that broke
cl-flac's CFFI calls
- cl-cron uses local time, not UTC: add utc-hour-to-local-hour
conversion so schedule hours fire at correct UTC times
(e.g. 12:00 UTC now fires at 15:00 local on UTC+3)
- Wrap each taglib field read in safe-tag with per-field error
handling so a type error in one field (e.g. album with non-simple
string) doesn't crash play-file or skip the track
- Use (coerce ... 'simple-string) instead of copy-seq for
guaranteed simple-string output from ensure-simple-string
- Prevent play-list thread death on scheduler playlist change:
drain-queue-into-remaining drains full scheduler queue at once,
updates loop-queue reference so repeat replays correct playlist,
top-level handler-case prevents thread from dying silently
- Fix taglib SIMPLE-ARRAY CHARACTER type errors:
ensure-simple-string coerces metadata strings and trims whitespace
- Fix FLAC::FILE slot unbound errors:
retry once with 200ms delay for transient init failures
- Improve playback state persistence:
save playlist path alongside track file so restart loads the
correct playlist instead of always falling back to stream-queue.m3u
- Startup now uses resume-from-saved-state to resolve saved playlist
and track position, falls back to stream-queue.m3u only if no state
- Enable loop-queue in play-list: repeats current playlist when tracks run out
- next-entry checks queue first so scheduler-queued playlists override repeat cycle
- Prevents silent stream / client reconnect loops between scheduled playlist changes
- Delay metadata/track-change notification by 1s after crossfade completes
- Log 'Loading next:' instead of 'Now playing:' during crossfade prep
- Add diagnostic logging: track duration check, crossfade trigger time
- harmony-load-playlist defaults to skip=nil: scheduler queues tracks
without interrupting current playback
- Save current track to .playback-state.lisp on each track change,
resume from saved position on restart
- Replace ~50 format-t debug statements in auth with log:info/log:warn
- Remove password hash logging for security
- Add .playback-state.lisp to .gitignore
- Use total listener count across all mounts instead of per-mount
(asteroid.lisp icecast-status API, stream-harmony.lisp now-playing)
- Fix recently-played.lisp: equal -> = for ParenScript string comparison
- Remove non-existent asteroid-shuffle.mp3 mount from recently-played JS
- Map all mount references to existing asteroid.mp3/asteroid.aac
Server-side fixes (stream-server.lisp):
- Add CORS preflight (OPTIONS) request handler for browser crossorigin audio
- AAC clients start from current buffer position instead of burst to avoid
ADTS frame alignment issues that caused browser decode errors
- Upgrade client stream error logging from debug to warn for diagnostics
- Add send-cors-preflight function with proper Access-Control headers
Frontend fixes (stream-player.lisp):
- Rewrite reconnect-stream to reuse existing audio element instead of
creating a new one, preserving browser user gesture context and preventing
NotAllowedError on autoplay after reconnect
- Unify stream config: both curated and shuffle channels use same mount
points (asteroid.mp3/asteroid.aac) since cl-streamer has a single pipeline
- Remove non-existent /asteroid-shuffle.mp3 mount reference that caused
404s and broken pipe cascade when switching to shuffle channel
- Map :low quality to same MP3 mount (asteroid-low.mp3 not yet available)
Note: Channel selector preserved for future multi-stream support.
Recently-played API works correctly; frontend rendering to investigate separately.
CORS fix (icy-protocol.lisp):
- Add Access-Control-Allow-Origin: * to stream response headers
- Browser audio player can now connect cross-origin (port 8080 -> 8000)
Auto-start (asteroid.lisp -main):
- Start cl-streamer pipeline automatically on boot
- Load stream-queue.m3u and begin playback immediately
- Wrapped in handler-case so streaming failure doesn't block web server
Mount names (stream-harmony.lisp):
- Renamed /stream.mp3 -> /asteroid.mp3, /stream.aac -> /asteroid.aac
- Matches existing frontend URLs, zero template changes needed
Icecast bypass (asteroid.lisp, listener-stats.lisp):
- Front page uses get-now-playing-stats instead of icecast-now-playing
- check-icecast-status returns cl-streamer status when pipeline is active
- check-liquidsoap-status returns N/A when using cl-streamer
- asteroid/icecast-status API returns cl-streamer data directly
- poll-and-store-stats uses cl-streamer listener counts directly
- Eliminates hanging HTTP requests to port 8000 for Icecast XML
Tested: full browser streaming working end-to-end
- Add taglib dependency to cl-streamer/harmony system
- Add read-audio-metadata: reads artist/title/album from FLAC/MP3 tags
- Add format-display-title: builds 'Artist - Title' from tags, falls back to filename
- Add update-all-mounts-metadata: updates ICY metadata on all mount points
- Defer metadata update during crossfade until fade completes (listeners hear correct track)
- Fix play-list wait loop: was nested inside crossfade conditional, first track never waited
- Remove filename-derived :title from test playlist (taglib reads real tags now)
- Fix FDK-AAC C shim: use proper OUT_BITSTREAM_DATA=3 constant and INT types
- Add frame-aligned PCM accumulation buffer to AAC encoder for clean output
- Add fdkaac_encode and fdkaac_close to C shim (all FDK-AAC calls in C)
- Implement crossfade between tracks (3s overlap, 2s fade-in/out)
- Tune buffer: 90% drain sleep for encoding headroom, 64KB burst-on-connect
- Add FFI bindings for new shim functions
- Rewrite README.org with full architecture docs and integration plan
- Broadcast buffer: single-producer multi-consumer ring buffer with
per-client read cursors. 32KB burst-on-connect for fast playback.
Never blocks producer (overwrites old data for slow clients).
- Sequential playlist: play-list runs tracks one at a time using
Harmony's on-end callback + condition variable for completion.
- ICY metadata: set-now-playing called on each track change.
- Fixed string vs pathname bug in harmony:play (etypecase mismatch).
- Debug logging for client disconnect diagnosis.
Verified: browser plays shuffled FLAC playlist via 128kbps MP3 stream.
Major changes:
- streaming-drain: Custom drain that captures PCM from Harmony's pack
buffer (raw IEEE 754 floats in unsigned-byte-8 array), converts to
signed-16 PCM via CFFI, encodes to MP3 via LAME, and writes to
stream server's ring buffer
- Fixed ring buffer deadlock: buffer-read/buffer-write held lock then
called buffer-available which tried to acquire same lock. Created
internal %buffer-available/%buffer-free-space without locking.
- Fixed ring buffer zero-length guard for unbound variable in finally
- Fixed sleep duration in drain: was dividing raw byte count by
samplerate, now correctly converts to frames first
- Added flexi-streams wrapper for bivalent HTTP socket I/O
- Exported all public API symbols from cl-streamer package
- Added test-stream.lisp end-to-end test script
Verified: Amon Tobin FLAC -> 128kbps MP3 stream at localhost:8000
file reports: MPEG ADTS, layer III, v1, 128kbps, 44.1kHz, JntStereo
- harmony-backend.lisp: Audio pipeline connecting Harmony to encoders
- Uses :dummy drain to capture PCM without audio output
- Reads from packer bip-buffer via request-read/finish-read API
- Encodes PCM to MP3 via LAME and feeds to stream server
All three systems compile cleanly:
cl-streamer, cl-streamer/encoder, cl-streamer/harmony
- Architecture document outlining Icecast/Liquidsoap replacement
- Core streaming server with ICY metadata protocol support
- Thread-safe ring buffer for audio data
- Mount point abstraction with metadata updates
- Multi-client connection handling
This is experimental groundwork for integrating with Harmony/cl-mixed
and playlisp/parsector for a pure CL streaming solution.
API-ENDPOINTS.org:
- Add documentation for all 90+ API endpoints
- Stream Queue, Scheduler, Track Requests, User Playlists
- Favorites, History, Profile, Account endpoints
- Frontend Partials, Utility endpoints
- Now covers 100% of define-api endpoints in codebase
PROJECT-OVERVIEW.org:
- Tidy up tables and formatting
The r-simple-rate library has a bug where rate limit counters can go
negative and never reset. This happens because the reset condition
only triggers when amount >= 0, so negative amounts are permanently
stuck.
This fix adds:
- cleanup-corrupted-rate-limits function to delete corrupted entries
- db:connected trigger to run cleanup automatically on startup
This prevents the 429 error loops that occurred when counters became
corrupted with large negative values.
Fix 429 errors caused by aggressive rate limiting on now-playing APIs.
Changed from 2-3 req/sec to 60 req/min for:
- asteroid/partial/now-playing
- asteroid/partial/now-playing-inline
- asteroid/partial/now-playing-json
- asteroid/channel-name
This fixes notifications not working and may resolve auto-reconnect issues.
- Implement Web Notifications API in ParenScript
- Add notification toggle button (🔔/🔕) to player frame
- Show 'Artist - Track' notification when track changes
- Store notification preference in localStorage
- Auto-close notifications after 5 seconds
- Click notification to focus browser window
- Add dropdown to admin template to choose sort order
- Update get-geo-stats to accept order-by parameter
- Update API endpoint to pass sort-by parameter
- Update ParenScript to read dropdown and pass to API
- Default sort is by minutes (engagement time)
- Add register-web-listener to now-playing-json API endpoint
This keeps listeners registered during continuous playback instead of
timing out after 5 minutes of inactivity
- Fix listen_minutes calculation to increment by listener_count per poll
Previously incremented by 1 regardless of how many listeners, now properly
tracks listener-minutes (1 minute per listener per 60s poll interval)
- Add migration 009 documenting the calculation fix
Glenn Thompson
Luis Pereira