Unless they are explicitly bound to loopback, which I thought was the default, but it is not. likely related to the interface between bridges and ip tables in the Linux kernel, but anyhow, get literal about the portmap interface address to prevent exposing the database to the entire internet. With thanks to the friendly heads up email from the German Federal Republic via Hetzner. |
||
|---|---|---|
| cl-streamer@23b5ee1e35 | ||
| config | ||
| data/sessions | ||
| docker | ||
| docs | ||
| migrations | ||
| parenscript | ||
| playlists | ||
| scripts | ||
| static | ||
| template | ||
| .dockerignore.asteroid | ||
| .gitignore | ||
| .gitmodules | ||
| AAC-STREAMING.org | ||
| Dockerfile.asteroid | ||
| LICENSE | ||
| Makefile | ||
| README.org | ||
| SPECTRUM-ANALYZER.org | ||
| TODO-next-features.org | ||
| TODO.org | ||
| analyze-performance.py | ||
| app-utils.lisp | ||
| asteroid.asd | ||
| asteroid.lisp | ||
| auth-routes.lisp | ||
| build-asteroid.lisp | ||
| comprehensive-performance-test.sh | ||
| conditions.lisp | ||
| database.lisp | ||
| design.org | ||
| dj-session.lisp | ||
| frontend-partials.lisp | ||
| limiter.lisp | ||
| listener-stats.lisp | ||
| module.lisp | ||
| parenscript-utils.lisp | ||
| playlist-management.lisp | ||
| playlist-scheduler.lisp | ||
| project-summary.org | ||
| run-all-tests.sh | ||
| setup-environment.lisp | ||
| simple-analysis.py | ||
| stream-control.lisp | ||
| stream-harmony.lisp | ||
| stream-media.lisp | ||
| template-utils.lisp | ||
| test-parenscript.lisp | ||
| test-ps-compile.lisp | ||
| test-server.sh | ||
| test-user-api.sh | ||
| track-requests.lisp | ||
| user-management.lisp | ||
| user-playlists.lisp | ||
| user-profile.lisp | ||
| users.lisp | ||
README.org
Asteroid Radio - Internet Radio Streaming Platform
- Overview
- Key Features
- Architecture
- Quick Start
- Playlist Scheduling
- Music Library Management
- User Management
- Player Modes
- API Endpoints
- Database
- Dependencies
- Resource Usage
- Testing
- Contributing
- Troubleshooting
- Documentation
- License
- Acknowledgments
- Credits
Overview
Asteroid Radio is a complete internet radio streaming platform built entirely in Common Lisp. It combines the Radiance web framework with an in-process audio streaming pipeline powered by Harmony, cl-mixed, and a custom streaming server called CL-Streamer. Everything runs in a single SBCL process — audio decoding, mixing, encoding, HTTP streaming, metadata, playlist scheduling, and the web frontend.
No external streaming services are required. The only external dependency is PostgreSQL for persistent data.
Project Links
- Repository: https://github.com/fade/asteroid
- IRC: #asteroid.music on irc.libera.chat
- Documentation: See
docs/directory for comprehensive guides
Key Features
Live Internet Radio Streaming
- Dual-format simultaneous output: 128kbps MP3 + 128kbps AAC
- Real-time audio decoding via Harmony (FLAC, MP3 via cl-mixed backends)
- Crossfade between tracks (3s overlap, configurable fade-in/fade-out)
- ICY metadata protocol for now-playing information in media players
- Burst-on-connect for instant playback start (~4s of buffered audio)
CL-Streamer — The Audio Pipeline
CL-Streamer is Asteroid Radio's own native Common Lisp audio streaming server, written from scratch as part of this project. It provides HTTP audio streaming with ICY metadata, dual-format encoding (MP3 + AAC), and real-time audio processing — replacing what previously required separate Icecast and Liquidsoap Docker containers. It lives in the cl-streamer/ directory and is loaded as an ASDF system within the Asteroid Radio process. See cl-streamer/README.org for full technical details.
Key components:
- streaming-drain — custom Harmony drain that intercepts mixed audio and feeds it to encoders instead of a sound card
- MP3 encoder — LAME via CFFI (
lame-ffi.lisp) - AAC encoder — FDK-AAC via a C shim that avoids SBCL signal handler conflicts (
fdkaac-shim.c,fdkaac-ffi.lisp) - Broadcast buffer — single-producer multi-consumer ring buffer per mount point; never blocks the encoder
- ICY metadata — SHOUTcast-compatible metadata interleaved in the stream
- HTTP server — usocket-based, serves
/asteroid.mp3and/asteroid.aacdirectly
Playlist Scheduling
- Cron-based playlist scheduler with configurable time slots (
playlist-scheduler.lisp) - M3U playlist files with
#EXTINFmetadata (playlists/directory) - Playback state persistence — saves current track/playlist position to disk (
.playback-state.lisp) - Automatic resume from saved state on restart
- Sequential playback with crossfade transitions
Music Library Management
- Database-backed track storage with metadata extraction
- Support for MP3 and FLAC formats (decoded by cl-mixed backends)
- Automatic metadata extraction using taglib (artist, title, album)
- Track search, filtering, sorting, and pagination
- Recursive directory scanning
Web Interface
- Radiance framework with CLIP templating
- Admin dashboard with stream status, listener stats, and library management
- Multiple player modes: inline, pop-out, and persistent frameset
- Live stream integration with embedded HTML5 audio player
- Responsive design for desktop and mobile
- Role-based access control (Admin/DJ/Listener)
- ParenScript-generated JavaScript (
parenscript/directory) - LASS-generated CSS (
static/asteroid.lass)
Listener Statistics
- In-process listener tracking (no external polling needed)
- Geo IP lookup with caching (
listener-stats.lisp) - Per-mount and aggregate listener counts
- GDPR-compliant IP hashing and data retention
Architecture
┌──────────────────────────────────────────────────────────────┐ │ SBCL Process │ │ │ │ ┌───────────┐ ┌──────────────┐ ┌───────────────────┐ │ │ │ Harmony │──▶│ streaming- │──▶│ MP3 Encoder │ │ │ │ (decode, │ │ drain │ │ (LAME FFI) │──▶│ /asteroid.mp3 │ │ crossfade,│ │ (float→s16 │ └───────────────────┘ │ │ │ mix) │ │ conversion) │ ┌───────────────────┐ │ │ └───────────┘ └──────────────┘──▶│ AAC Encoder │ │ │ ▲ │ (FDK-AAC C shim) │──▶│ /asteroid.aac │ │ └───────────────────┘ │ │ ┌───────────┐ ┌───────────────────┐ │ │ │ Playlist │ ICY metadata ────▶│ CL-Streamer │ │ │ │ Scheduler │ Listener stats ◀──│ HTTP Server │ │ │ │ (cl-cron) │ │ (usocket) │ │ │ └───────────┘ └───────────────────┘ │ │ │ │ │ ┌───────────┐ ┌──────────────┐ ┌───────────────────┐ │ │ │ Radiance │──▶│ Admin │ │ PostgreSQL │ │ │ │ Web Server │ │ Dashboard │ │ (external) │ │ │ │ (port 8080)│ │ + Player UI │ │ │ │ │ └───────────┘ └──────────────┘ └───────────────────┘ │ └──────────────────────────────────────────────────────────────┘
How Audio Flows
- Decode — Harmony loads a FLAC or MP3 file via cl-mixed-flac or cl-mixed-mpg123
- Mix — during crossfade, two voices play simultaneously through Harmony's mixer
- Drain — the custom
streaming-drainreads interleaved float samples from the pack buffer - Convert — floats are converted to signed 16-bit PCM via CFFI
- Encode — PCM is fed to LAME (MP3) and FDK-AAC (AAC) encoders in parallel
- Buffer — encoded bytes are written to per-mount broadcast ring buffers
- Serve — HTTP clients read from their position in the ring buffer with ICY metadata interleaving
File Structure
asteroid/
├── asteroid.lisp # Main server, Radiance routes, API endpoints
├── asteroid.asd # ASDF system definition
├── stream-harmony.lisp # Harmony/CL-Streamer integration, playback state
├── playlist-scheduler.lisp # Cron-based playlist scheduling
├── stream-control.lisp # Stream queue management
├── listener-stats.lisp # Listener statistics and geo IP
├── user-management.lisp # User administration
├── playlist-management.lisp # Playlist operations
├── user-playlists.lisp # User playlist features
├── user-profile.lisp # User profile management
├── track-requests.lisp # Track request system
├── auth-routes.lisp # Authentication routes
├── database.lisp # Database schema and helpers
├── Makefile # Build: compiles SBCL image
├── cl-streamer/ # Audio streaming subsystem
│ ├── cl-streamer.asd # ASDF system definition
│ ├── harmony-backend.lisp # Harmony integration, crossfade, play-list/play-file
│ ├── stream-server.lisp # HTTP streaming server (usocket)
│ ├── buffer.lisp # Broadcast ring buffer
│ ├── encoder.lisp # MP3 encoder (LAME FFI)
│ ├── aac-encoder.lisp # AAC encoder with frame accumulation
│ ├── lame-ffi.lisp # CFFI bindings for libmp3lame
│ ├── fdkaac-ffi.lisp # CFFI bindings for FDK-AAC (via shim)
│ ├── fdkaac-shim.c # C shim for FDK-AAC (avoids SBCL signal conflicts)
│ ├── libfdkaac-shim.so # Compiled shim shared library
│ ├── icy-protocol.lisp # ICY metadata encoding
│ └── README.org # Full CL-Streamer documentation
├── playlists/ # M3U playlist files for the scheduler
├── template/ # CLIP HTML templates
├── parenscript/ # ParenScript JavaScript sources
├── static/ # CSS (LASS) and assets
├── migrations/ # PostgreSQL schema migrations
├── docker/ # Docker config (PostgreSQL only)
├── docs/ # Comprehensive documentation
└── music/ # Music library symlink
Quick Start
Prerequisites
- SBCL — Steel Bank Common Lisp
- Quicklisp — with Ultralisp distribution (for Harmony, cl-mixed)
- PostgreSQL — running instance (Docker or native)
- System libraries:
libmp3lame,libfdk-aac,libflac,libmpg123,libtagc
Building the FDK-AAC Shim
CL-Streamer uses a thin C shim to call FDK-AAC, avoiding SBCL signal handler conflicts that cause recursive SIGSEGV when calling FDK-AAC directly via CFFI. The compiled libfdkaac-shim.so is included in the repository, but if you need to rebuild it (e.g. after updating libfdk-aac):
cd cl-streamer
gcc -shared -fPIC -o libfdkaac-shim.so fdkaac-shim.c -lfdk-aac
The shim is loaded automatically at ASDF system load time from the cl-streamer/ directory — no installation or LD_LIBRARY_PATH configuration needed.
Build and Run
# Clone repository
git clone https://github.com/fade/asteroid
cd asteroid
# Start PostgreSQL (if using Docker)
cd docker && docker compose up -d postgres && cd ..
# Build the SBCL image
make
# Run (set ASTEROID_STREAM_URL to the stream server address)
ASTEROID_STREAM_URL=http://localhost:8000 ./asteroid
On startup, Asteroid will:
- Start the Radiance web server on port 8080
- Start the CL-Streamer HTTP server on port 8000
- Initialise the Harmony audio pipeline with MP3 + AAC encoders
- Connect to PostgreSQL and load the playlist schedule
- Resume playback from saved state (if
.playback-state.lispexists) - Begin streaming audio on
/asteroid.mp3and/asteroid.aac
Access Points
- Web Interface: http://localhost:8080/
- Admin Panel: http://localhost:8080/admin
- MP3 Stream: http://localhost:8000/asteroid.mp3 (128kbps)
- AAC Stream: http://localhost:8000/asteroid.aac (128kbps)
Playlist Scheduling
Playlists are M3U files in the playlists/ directory. The scheduler (playlist-scheduler.lisp) uses cl-cron to switch playlists at configured times.
Schedule Configuration
The schedule is stored in the database and loaded on startup. Example schedule:
| Time (UTC) | Playlist | Description |
|---|---|---|
| 00:00 | midnight-ambient.m3u | Deep, dark ambient |
| 06:00 | morning-drift.m3u | Lighter, awakening ambient |
| 12:00 | afternoon-orbit.m3u | Mid-energy floating ambient |
| 13:00 | underworld-and-friends | Techno, electro, progressive |
| 18:00 | evening-descent.m3u | Winding down transitional |
M3U Format
Playlists use #EXTINF metadata and /app/music/ prefixed paths (rewritten to local library path at load time):
#EXTM3U
#PLAYLIST:Underworld & Friends
#EXTINF:-1,Underworld - Born Slippy (Nuxx)
/app/music/Underworld - Second Toughest In The Infants (flac)/Second Toughest In The Infants (CD2)/01 Born Slippy (Nuxx).flac
#EXTINF:-1,Orbital - Halcyon + On + On
/app/music/Orbital/1993 - Orbital - Orbital 2 (Brown Album - TRUCD2, 828 386.2)/00. Halcyon + On + On.mp3
Playback State Persistence
The current track and playlist position are saved to .playback-state.lisp periodically. On restart, playback resumes from the saved position rather than restarting the playlist from the beginning.
Music Library Management
Adding Music
- Place MP3/FLAC files in the music library directory (symlinked from
music/library/) - Navigate to the admin panel at
http://localhost:8080/admin - Click "Scan Library" to index new tracks
- Metadata (artist, title, album) is automatically extracted via taglib
Supported Formats
- FLAC — decoded by cl-mixed-flac (via libflac)
- MP3 — decoded by cl-mixed-mpg123 (via libmpg123)
Path Mapping
In Docker deployments, music paths use /app/music/ prefix. In local development, music/library/ is a symlink to the actual music directory. The M3U playlist loader rewrites paths accordingly.
User Management
Roles
- Admin: Full system access, user management, stream control, skip tracks
- DJ: Content management, playlist creation, library access
- Listener: Basic playback and personal playlists
Default Credentials
- Username:
admin - Password:
asteroid123 - ⚠️ Change default password after first login
Player Modes
Inline Player
- Embedded in web pages
- HTML5 audio with stream auto-reconnect
Pop-Out Player
- Standalone player window
- Independent from main browser window
Frameset Player
- Bottom-frame persistent player
- Audio continues during site navigation
- Seamless listening experience
API Endpoints
Status & Streaming
GET /api/asteroid/status- Server statusGET /api/asteroid/icecast-status- Stream status (now-playing, listeners; name kept for frontend compatibility)POST /api/asteroid/stream/skip- Skip current track (admin)
Track Management
GET /api/asteroid/tracks- List all tracksGET /api/asteroid/admin/tracks- Admin track listingPOST /api/asteroid/admin/scan-library- Scan music library
Playlist Management
GET /api/asteroid/playlists- List user playlistsPOST /api/asteroid/playlists/create- Create playlistGET /api/asteroid/playlists/get- Get playlist detailsPOST /api/asteroid/playlists/add-track- Add track to playlist
Stream Queue Control (Admin)
GET /api/asteroid/stream/queue- Get broadcast queuePOST /api/asteroid/stream/queue/add- Add track to queuePOST /api/asteroid/stream/queue/remove- Remove from queuePOST /api/asteroid/stream/queue/clear- Clear queue
See docs/API-ENDPOINTS.org for complete API documentation.
Database
PostgreSQL
- Primary database for tracks, users, playlists, sessions, schedule, listener stats
- Docker container or native installation
- Schema managed via migrations in
migrations/directory - Connection configured in
config/radiance-postgres.lisp - See
docs/POSTGRESQL-SETUP.orgfor setup details
Dependencies
Lisp Dependencies
radiance- Web frameworkharmony- Audio framework (decode, mix, effects)cl-mixed/cl-mixed-flac/cl-mixed-mpg123- Low-level audio mixing and decodingr-clip- CLIP templatinglass- CSS preprocessingparenscript- Lisp-to-JavaScript compilercl-json- JSON handlingalexandria- Common Lisp utilitiesbordeaux-threads- Portable threadinglocal-time- Time handlingtaglib- Audio metadata extractioncl-cron- Cron-style job schedulingcffi- C foreign function interfaceusocket- Socket networkingi-postmodern- PostgreSQL interface (Radiance)ironclad- Cryptographic hashing (passwords, IP hashing)log4cl- Loggingslynk- Sly/SLIME interactive development
System Libraries
libmp3lame- MP3 encoding (LAME)libfdk-aac- AAC encoding (FDK-AAC, accessed via C shim)libflac- FLAC decodinglibmpg123- MP3 decodinglibtagc- Audio metadata reading (taglib C bindings)
Infrastructure
- PostgreSQL - only external service required
- Docker - optional, for running PostgreSQL
Resource Usage
Running on a single SBCL process:
- RAM: ~320MB RSS (real-time audio decode + dual MP3/AAC encode)
- CPU: ~8% total (Harmony thread is ~5%)
- Threads: ~18
Testing
Automated Test Suite
# Run comprehensive tests
./test-server.sh
# Verbose mode
./test-server.sh -v
Test Coverage
- 25+ automated tests
- API endpoint validation
- HTML page rendering
- Static file serving
- JSON response format
- Authentication flows
Contributing
Development Workflow
- Fork the repository
- Create a feature branch
- Make your changes
- Run test suite
- Submit pull request
Interactive Development
Asteroid starts a Slynk server on port 4009. Connect from Emacs/Sly for live REPL access to the running system — inspect streams, modify playlists, debug audio issues without restarting.
Community
- IRC: #asteroid.music on irc.libera.chat
- Issues: GitHub issue tracker
- Discussions: GitHub discussions
Core Team
- Brian O'Reilly (Fade) - Project founder
- Glenn Thompson (glenneth) - Core developer
- Luis Pereira - UI/UX
Troubleshooting
Stream Not Playing
- Check that the process is running and CL-Streamer started (look for "CL-Streamer started on port 8000" in logs)
- Test stream URLs:
curl -I http://localhost:8000/asteroid.mp3 - Check for audio file errors in the log (missing files, unsupported formats)
- Verify music library symlink:
ls -la music/library/
SBCL Pathname Issues
File paths containing square brackets (e.g. [WEB FLAC]) can cause SIMPLE-ARRAY CHARACTER errors because SBCL's pathname function interprets brackets as wildcard patterns. CL-Streamer uses sb-ext:parse-native-namestring to avoid this.
Database Issues
- Check PostgreSQL is running:
docker compose ps(if using Docker) - Verify connection in
config/radiance-postgres.lisp - Review application logs for connection errors
Playback State
- Saved to
.playback-state.lispin the project root - Delete this file to force a fresh start from the scheduled playlist
- Check logs for "Resuming after track…" on startup
For detailed troubleshooting, see documentation in docs/ directory.
Documentation
Comprehensive documentation available in the docs/ directory:
- README.org - Documentation index
- PROJECT-OVERVIEW.org - Architecture and features
- PROJECT-HISTORY.org - Development timeline and milestones
- INSTALLATION.org - Complete installation guide
- DEVELOPMENT.org - Developer setup and guidelines
- API-ENDPOINTS.org - REST API reference
- STREAM-CONTROL.org - Stream queue management
- USER-MANAGEMENT-SYSTEM.org - User administration
- PLAYLIST-SYSTEM.org - Playlist functionality
- TESTING.org - Automated testing guide
- POSTGRESQL-SETUP.org - Database setup
See also cl-streamer/README.org for CL-Streamer technical documentation.
License
See LICENSE file for details.
Acknowledgments
Built with:
- SBCL (Steel Bank Common Lisp)
- Radiance web framework (Shinmera)
- Harmony audio framework (Shinmera)
- cl-mixed audio mixing library (Shinmera)
- LAME MP3 encoder
- FDK-AAC AAC encoder
Special thanks to Yukari Hafner (Shinmera) for Harmony, cl-mixed, and Radiance, and to the Common Lisp community.