glenn
/
asteroid
mirror of https://github.com/glenneth1/asteroid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
asteroid/docs/CL-STREAMING-ARCHITECTURE.org

7.2 KiB
Raw Blame History

CL-Native Streaming Architecture

Overview

This document outlines the architecture for replacing Icecast and Liquidsoap with a pure Common Lisp streaming solution for Asteroid Radio.

Goals

  • Eliminate external dependencies on Icecast and Liquidsoap
  • Leverage existing CL audio ecosystem (Harmony, cl-mixed, playlisp)
  • Maintain compatibility with current Asteroid features
  • Enable tighter integration between web app and streaming engine

Current Architecture (Icecast/Liquidsoap) 

┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐
│   Asteroid      │────▶│   Liquidsoap    │────▶│    Icecast      │
│   (Radiance)    │     │   (Source)      │     │   (Server)      │
└─────────────────┘     └─────────────────┘     └─────────────────┘
        │                       │                       │
        │ Telnet commands       │ Audio stream          │ HTTP streams
        │ Playlist updates      │ (SOURCE protocol)     │ to listeners
        ▼                       ▼                       ▼
   playlists/            Decode/Encode           /asteroid.mp3
   stream-queue.m3u      Crossfade               /asteroid.aac

Proposed Architecture (CL-Native) 

┌─────────────────────────────────────────────────────────────────┐
│                         ASTEROID                                 │
│  ┌───────────────────────────────────────────────────────────┐  │
│  │                    CL-STREAMER                            │  │
│  │  ┌─────────────┐  ┌─────────────┐  ┌─────────────────┐   │  │
│  │  │  Playlist   │  │   Audio     │  │  HTTP Stream    │   │  │
│  │  │  Engine     │──▶│  Pipeline   │──▶│  Server         │   │  │
│  │  │ (playlisp)  │  │ (Harmony)   │  │ (ICY protocol)  │   │  │
│  │  └─────────────┘  └─────────────┘  └─────────────────┘   │  │
│  │        │                │                   │             │  │
│  │        ▼                ▼                   ▼             │  │
│  │   Parse M3U        Decode/Mix/         Multi-client      │  │
│  │   Track queue      Encode              streaming         │  │
│  └───────────────────────────────────────────────────────────┘  │
│                                                                  │
│  Existing: Web UI, API, Scheduler, User features                │
└─────────────────────────────────────────────────────────────────┘

Components

1. Playlist Engine (playlisp/parsector)

Already implemented by Fade. Provides:

  • M3U parsing into CLOS objects
  • Track metadata (title, artist, duration, path)
  • Arithmetic expressions in duration fields
  • Playlist construction API

Location: ~/SourceCode/playlisp

2. Audio Pipeline (Harmony + cl-mixed)

Shinmera's audio system provides:

Decoding (via cl-mixed extensions)

  • cl-mixed-mpg123 - MP3 decoding
  • cl-flac - FLAC decoding
  • cl-vorbis - OGG/Vorbis decoding
  • cl-opus - OGG/Opus decoding

Mixing

  • Multiple mixers (music, sfx, etc.)
  • Voice management
  • Effects chain (filters, EQ, etc.)

Crossfading

  • Environment system for horizontal mixing
  • Transition between segments with configurable fade times

Output

  • Platform drains (PulseAudio, CoreAudio, WASAPI)
  • Need: Network/buffer output for streaming

3. Audio Encoding (TO BE IMPLEMENTED)

Need to encode PCM audio to streaming formats:

Options

  • cl-lame - FFI bindings to LAME (MP3 encoding)
  • cl-fdkaac - FFI bindings to FDK-AAC (AAC encoding)
  • cl-opus - May support encoding (verify)

Implementation Notes

  • Harmony outputs PCM samples
  • Need to capture these and encode in real-time
  • Ring buffer between mixer output and encoder input

4. HTTP Stream Server (TO BE IMPLEMENTED)

Replace Icecast with CL-native HTTP streaming:

Requirements

  • HTTP/1.1 chunked transfer encoding
  • ICY metadata protocol (track titles in stream)
  • Multiple mount points (/asteroid.mp3, /asteroid.aac)
  • Concurrent client connections
  • Listener statistics

ICY Protocol 

Client request:
GET /stream HTTP/1.1
Icy-MetaData: 1

Server response:
HTTP/1.1 200 OK
Content-Type: audio/mpeg
icy-metaint: 16000

[audio data - 16000 bytes]
[metadata block]
[audio data - 16000 bytes]
[metadata block]
...

Implementation Options

  • Build on Hunchentoot (acceptor + handler)
  • Build on usocket directly (more control)
  • Use Chunga for chunked encoding

Dependencies

Existing (in Quicklisp/Shirakumo dist)

  • harmony
  • cl-mixed
  • cl-mixed-mpg123
  • cl-flac
  • chunga
  • usocket
  • bordeaux-threads

From Fade's repos

  • playlisp (~/SourceCode/playlisp)
  • parsector (~/SourceCode/parsector)

To Be Created/Found

  • MP3 encoder bindings (LAME)
  • AAC encoder bindings (FDK-AAC) - optional

Implementation Phases

Phase 1: Proof of Concept

  • Load Harmony and play audio files from playlist
  • Capture PCM output to buffer
  • Encode buffer to MP3 using LAME FFI
  • Serve single HTTP stream to one client

Phase 2: Core Streaming

  • Implement ICY metadata injection
  • Multi-client connection handling
  • Ring buffer for audio data
  • Mount point abstraction

Phase 3: Integration

  • Replace liquidsoap-command calls with direct CL calls
  • Integrate with existing playlist scheduler
  • Crossfading between tracks
  • Listener statistics

Phase 4: Feature Parity

  • Multiple stream formats (MP3, AAC)
  • Multiple bitrates
  • Admin controls (skip, reload)
  • YP directory registration

Open Questions

  1. Should cl-streamer be a separate ASDF system or part of asteroid?
  2. How to handle the transition period (run both systems)?
  3. Performance: Can CL handle 100+ concurrent listeners?
  4. Licensing: LAME is LGPL, FDK-AAC has patent issues

References