199 lines
7.2 KiB
Org Mode
199 lines
7.2 KiB
Org Mode
#+TITLE: CL-Native Streaming Architecture
|
|
#+AUTHOR: Glenn Thompson
|
|
#+DATE: 2026-03-03
|
|
#+OPTIONS: toc:2
|
|
|
|
* Overview
|
|
|
|
This document outlines the architecture for replacing Icecast and Liquidsoap
|
|
with a pure Common Lisp streaming solution for Asteroid Radio.
|
|
|
|
** Goals
|
|
|
|
- Eliminate external dependencies on Icecast and Liquidsoap
|
|
- Leverage existing CL audio ecosystem (Harmony, cl-mixed, playlisp)
|
|
- Maintain compatibility with current Asteroid features
|
|
- Enable tighter integration between web app and streaming engine
|
|
|
|
** Current Architecture (Icecast/Liquidsoap)
|
|
|
|
#+begin_example
|
|
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
|
|
│ Asteroid │────▶│ Liquidsoap │────▶│ Icecast │
|
|
│ (Radiance) │ │ (Source) │ │ (Server) │
|
|
└─────────────────┘ └─────────────────┘ └─────────────────┘
|
|
│ │ │
|
|
│ Telnet commands │ Audio stream │ HTTP streams
|
|
│ Playlist updates │ (SOURCE protocol) │ to listeners
|
|
▼ ▼ ▼
|
|
playlists/ Decode/Encode /asteroid.mp3
|
|
stream-queue.m3u Crossfade /asteroid.aac
|
|
#+end_example
|
|
|
|
** Proposed Architecture (CL-Native)
|
|
|
|
#+begin_example
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ ASTEROID │
|
|
│ ┌───────────────────────────────────────────────────────────┐ │
|
|
│ │ CL-STREAMER │ │
|
|
│ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────────┐ │ │
|
|
│ │ │ Playlist │ │ Audio │ │ HTTP Stream │ │ │
|
|
│ │ │ Engine │──▶│ Pipeline │──▶│ Server │ │ │
|
|
│ │ │ (playlisp) │ │ (Harmony) │ │ (ICY protocol) │ │ │
|
|
│ │ └─────────────┘ └─────────────┘ └─────────────────┘ │ │
|
|
│ │ │ │ │ │ │
|
|
│ │ ▼ ▼ ▼ │ │
|
|
│ │ Parse M3U Decode/Mix/ Multi-client │ │
|
|
│ │ Track queue Encode streaming │ │
|
|
│ └───────────────────────────────────────────────────────────┘ │
|
|
│ │
|
|
│ Existing: Web UI, API, Scheduler, User features │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
#+end_example
|
|
|
|
* Components
|
|
|
|
** 1. Playlist Engine (playlisp/parsector)
|
|
|
|
Already implemented by Fade. Provides:
|
|
|
|
- M3U parsing into CLOS objects
|
|
- Track metadata (title, artist, duration, path)
|
|
- Arithmetic expressions in duration fields
|
|
- Playlist construction API
|
|
|
|
Location: =~/SourceCode/playlisp=
|
|
|
|
** 2. Audio Pipeline (Harmony + cl-mixed)
|
|
|
|
Shinmera's audio system provides:
|
|
|
|
*** Decoding (via cl-mixed extensions)
|
|
- =cl-mixed-mpg123= - MP3 decoding
|
|
- =cl-flac= - FLAC decoding
|
|
- =cl-vorbis= - OGG/Vorbis decoding
|
|
- =cl-opus= - OGG/Opus decoding
|
|
|
|
*** Mixing
|
|
- Multiple mixers (music, sfx, etc.)
|
|
- Voice management
|
|
- Effects chain (filters, EQ, etc.)
|
|
|
|
*** Crossfading
|
|
- Environment system for horizontal mixing
|
|
- Transition between segments with configurable fade times
|
|
|
|
*** Output
|
|
- Platform drains (PulseAudio, CoreAudio, WASAPI)
|
|
- Need: Network/buffer output for streaming
|
|
|
|
** 3. Audio Encoding (TO BE IMPLEMENTED)
|
|
|
|
Need to encode PCM audio to streaming formats:
|
|
|
|
*** Options
|
|
- =cl-lame= - FFI bindings to LAME (MP3 encoding)
|
|
- =cl-fdkaac= - FFI bindings to FDK-AAC (AAC encoding)
|
|
- =cl-opus= - May support encoding (verify)
|
|
|
|
*** Implementation Notes
|
|
- Harmony outputs PCM samples
|
|
- Need to capture these and encode in real-time
|
|
- Ring buffer between mixer output and encoder input
|
|
|
|
** 4. HTTP Stream Server (TO BE IMPLEMENTED)
|
|
|
|
Replace Icecast with CL-native HTTP streaming:
|
|
|
|
*** Requirements
|
|
- HTTP/1.1 chunked transfer encoding
|
|
- ICY metadata protocol (track titles in stream)
|
|
- Multiple mount points (/asteroid.mp3, /asteroid.aac)
|
|
- Concurrent client connections
|
|
- Listener statistics
|
|
|
|
*** ICY Protocol
|
|
#+begin_example
|
|
Client request:
|
|
GET /stream HTTP/1.1
|
|
Icy-MetaData: 1
|
|
|
|
Server response:
|
|
HTTP/1.1 200 OK
|
|
Content-Type: audio/mpeg
|
|
icy-metaint: 16000
|
|
|
|
[audio data - 16000 bytes]
|
|
[metadata block]
|
|
[audio data - 16000 bytes]
|
|
[metadata block]
|
|
...
|
|
#+end_example
|
|
|
|
*** Implementation Options
|
|
- Build on Hunchentoot (acceptor + handler)
|
|
- Build on usocket directly (more control)
|
|
- Use Chunga for chunked encoding
|
|
|
|
* Dependencies
|
|
|
|
** Existing (in Quicklisp/Shirakumo dist)
|
|
- harmony
|
|
- cl-mixed
|
|
- cl-mixed-mpg123
|
|
- cl-flac
|
|
- chunga
|
|
- usocket
|
|
- bordeaux-threads
|
|
|
|
** From Fade's repos
|
|
- playlisp (~/SourceCode/playlisp)
|
|
- parsector (~/SourceCode/parsector)
|
|
|
|
** To Be Created/Found
|
|
- MP3 encoder bindings (LAME)
|
|
- AAC encoder bindings (FDK-AAC) - optional
|
|
|
|
* Implementation Phases
|
|
|
|
** Phase 1: Proof of Concept
|
|
- [ ] Load Harmony and play audio files from playlist
|
|
- [ ] Capture PCM output to buffer
|
|
- [ ] Encode buffer to MP3 using LAME FFI
|
|
- [ ] Serve single HTTP stream to one client
|
|
|
|
** Phase 2: Core Streaming
|
|
- [ ] Implement ICY metadata injection
|
|
- [ ] Multi-client connection handling
|
|
- [ ] Ring buffer for audio data
|
|
- [ ] Mount point abstraction
|
|
|
|
** Phase 3: Integration
|
|
- [ ] Replace liquidsoap-command calls with direct CL calls
|
|
- [ ] Integrate with existing playlist scheduler
|
|
- [ ] Crossfading between tracks
|
|
- [ ] Listener statistics
|
|
|
|
** Phase 4: Feature Parity
|
|
- [ ] Multiple stream formats (MP3, AAC)
|
|
- [ ] Multiple bitrates
|
|
- [ ] Admin controls (skip, reload)
|
|
- [ ] YP directory registration
|
|
|
|
* Open Questions
|
|
|
|
1. Should cl-streamer be a separate ASDF system or part of asteroid?
|
|
2. How to handle the transition period (run both systems)?
|
|
3. Performance: Can CL handle 100+ concurrent listeners?
|
|
4. Licensing: LAME is LGPL, FDK-AAC has patent issues
|
|
|
|
* References
|
|
|
|
- [[https://codeberg.org/shirakumo/harmony][Harmony - CL Sound System]]
|
|
- [[https://codeberg.org/shirakumo/cl-mixed][cl-mixed - Audio Mixing Library]]
|
|
- [[https://github.com/fade/playlisp][playlisp - M3U Parser]]
|
|
- [[https://github.com/fade/parsector][parsector - Parser Combinators]]
|
|
- [[https://cast.readme.io/docs/icy][ICY Protocol Documentation]]
|
|
- [[https://www.icecast.org/docs/][Icecast Documentation]]
|