asteroid/README.org

#+TITLE: Asteroid Radio - Internet Streaming Platform
#+AUTHOR: Glenn Thompson & Brian O'Reilly (Fade)
#+DATE: 2025-10-16

* Overview

Asteroid Radio is a modern, web-based music streaming platform built with Common Lisp and the Radiance framework. It provides a complete internet radio streaming system with live broadcasting, user management, playlist control, and professional audio processing.

* Key Features

** Live Internet Radio Streaming
- Multiple streaming formats: AAC 96kbps (high efficiency), MP3 128kbps (standard), MP3 64kbps (low bandwidth)
- Professional audio processing with ReplayGain for consistent volume without pumping
- Smooth crossfading between tracks (5 second transitions)
- Dynamic compression to prevent clipping
- Icecast2 streaming server integration
- Liquidsoap audio pipeline with watch-based playlist reloading

** Stream Queue Control System
- Curated stream queue management (play tracks in specific order)
- Add tracks to end of queue or as "next to play"
- Add entire playlists to stream queue
- Real-time queue reordering
- Automatic fallback to random playback when queue is empty
- Stream history tracking
- Admin-only queue control via REST API

** Music Library Management
- PostgreSQL-backed track storage with metadata extraction
- Support for MP3, FLAC, OGG, and WAV formats
- Automatic metadata extraction using taglib
- Track search, filtering, and sorting capabilities
- Pagination support for large libraries
- Individual track streaming on-demand

** User Management System
- User registration and authentication
- Role-based access control (admin/user)
- User profiles and session management
- Personal playlist creation and management
- Admin interface for user administration

** Web Interface
- RADIANCE framework with CLIP templating
- Modern admin dashboard with JavaScript controls
- Web player with HTML5 audio controls
- Live stream integration with embedded player
- Dark hacker-themed aesthetic with VT323 font
- Responsive design for desktop and mobile

** Network Broadcasting
- Docker-based streaming infrastructure
- WSL-compatible networking for internal network access
- Professional streaming URLs for media players
- Multi-listener support via Icecast2
- Telnet control interface for live DJ operations

* Architecture Changes

** Framework Migration
- Migrated from Hunchentoot to RADIANCE web framework
- Implemented proper domain routing (=/asteroid/=)
- CLIP templating system for dynamic content
- Database abstraction layer for track storage

** Streaming Stack
- *Icecast2*: Streaming server (port 8000)
- *Liquidsoap*: Audio processing and streaming pipeline with telnet control (port 1234)
- *RADIANCE*: Web server and API (port 8080)
- *PostgreSQL*: User accounts, track metadata, and playlist storage
- *Docker Compose*: Container orchestration for streaming services

** File Structure
#+BEGIN_SRC
asteroid/
├── asteroid.lisp              # Main server with RADIANCE routes
├── asteroid.asd               # System definition with dependencies
├── stream-control.lisp        # Stream queue management system
├── stream-media.lisp          # Media streaming and track management
├── user-management.lisp       # User authentication and profiles
├── playlist-management.lisp   # User playlist operations
├── auth-routes.lisp           # Authentication endpoints
├── stream-queue.m3u           # Generated stream queue playlist
├── docker/                    # Docker streaming infrastructure
│   ├── docker-compose.yml     # Container orchestration
│   ├── asteroid-radio-docker.liq  # Liquidsoap configuration
│   └── start.sh               # Container startup script
├── template/                  # CLIP HTML templates
│   ├── front-page.chtml       # Main page with live stream
│   ├── admin.chtml            # Admin dashboard with queue controls
│   ├── player.chtml           # Web player interface
│   └── login.chtml            # User authentication
├── static/                    # Frontend assets
│   ├── asteroid.lass          # LASS stylesheet source
│   ├── asteroid.css           # Compiled CSS
│   └── js/
│       ├── admin.js           # Admin interface controls
│       └── player.js          # Web player functionality
├── docs/                      # Documentation
│   ├── STREAM-CONTROL.org     # Queue management guide
│   ├── API-REFERENCE.org      # Complete API documentation
│   ├── USER-MANAGEMENT-SYSTEM.org  # User system guide
│   └── INSTALLATION.org       # Setup instructions
└── music/                     # Music library
    └── library/               # Music files
#+END_SRC

* Track Upload Workflow

** Current Implementation (Manual Upload)
1. *Copy files to staging*: Place MP3/FLAC files in =music/incoming/=
2. *Access admin panel*: Navigate to =http://[IP]:8080/asteroid/admin=
3. *Process files*: Click "Copy Files from Incoming" button
4. *Database update*: Files are moved to =music/library/= and metadata extracted
5. *Automatic playlist*: =playlist.m3u= is regenerated for streaming

** File Processing Steps
1. Files copied from =music/incoming/= to =music/library/=
2. Metadata extracted using taglib (title, artist, album, duration, bitrate)
3. Database record created with file path and metadata
4. Playlist file updated for Liquidsoap streaming
5. Files immediately available for on-demand streaming

** Supported Formats
- *MP3*: Primary format, best compatibility
- *FLAC*: Lossless audio, high quality
- *OGG*: Open source format
- *WAV*: Uncompressed audio

* Icecast2 Integration

** Configuration
- *Server*: localhost:8000
- *Mount point*: =/asteroid.mp3=
- *Password*: =b3l0wz3r0= (configured in Liquidsoap)
- *Format*: MP3 128kbps stereo

** Installation (Ubuntu/Debian)
#+BEGIN_SRC bash
sudo apt update
sudo apt install icecast2
sudo systemctl enable icecast2
sudo systemctl start icecast2
#+END_SRC

** Stream Access
- *Direct URL*: =http://[IP]:8000/asteroid.mp3=
- *Admin interface*: =http://[IP]:8000/admin/=
- *Statistics*: =http://[IP]:8000/status.xsl=

* Liquidsoap Integration

** Configuration File: =docker/asteroid-radio-docker.liq=
#+BEGIN_SRC liquidsoap
#!/usr/bin/liquidsoap

# Allow running as root in Docker
set("init.allow_root", true)
log.level.set(4)

# Enable telnet server for remote control
settings.server.telnet.set(true)
settings.server.telnet.port.set(1234)
settings.server.telnet.bind_addr.set("0.0.0.0")

# Create playlist source from managed stream queue
radio = playlist(
  mode="normal",  # Play in order (not randomized)
  reload=5,       # Check for playlist updates every 5 seconds
  reload_mode="watch",  # Watch file for changes
  "/app/stream-queue.m3u"
)

# Fallback to directory scan if queue is empty
radio_fallback = playlist.safe(
  mode="randomize",
  reload=3600,
  "/app/music/"
)

radio = fallback(track_sensitive=false, [radio, radio_fallback])

# Use ReplayGain for consistent volume without pumping
radio = amplify(1.0, override="replaygain", radio)

# Add smooth crossfade between tracks (5 seconds)
radio = crossfade(
  duration=5.0,
  fade_in=3.0,
  fade_out=3.0,
  radio
)

# Add compressor to prevent clipping
radio = compress(
  ratio=3.0,
  threshold=-15.0,
  attack=50.0,
  release=400.0,
  radio
)

# Output to Icecast2 in multiple formats
output.icecast(%mp3(bitrate=128), host="icecast", port=8000,
  password="H1tn31EhsyLrfRmo", mount="asteroid.mp3",
  name="Asteroid Radio", radio)

output.icecast(%fdkaac(bitrate=96), host="icecast", port=8000,
  password="H1tn31EhsyLrfRmo", mount="asteroid.aac",
  name="Asteroid Radio (AAC)", radio)

output.icecast(%mp3(bitrate=64), host="icecast", port=8000,
  password="H1tn31EhsyLrfRmo", mount="asteroid-low.mp3",
  name="Asteroid Radio (Low Quality)", radio)
#+END_SRC

** Installation (Ubuntu/Debian)
#+BEGIN_SRC bash
sudo apt update
sudo apt install liquidsoap
#+END_SRC

** Features
- *Stream queue control*: Curated playlist with watch-based reloading (5 second updates)
- *Smart fallback*: Random playback when queue is empty
- *ReplayGain processing*: Consistent volume without pumping artifacts
- *Crossfading*: Smooth 5-second transitions between tracks
- *Dynamic compression*: Prevents clipping and maintains audio quality
- *Multi-format output*: AAC 96kbps, MP3 128kbps, MP3 64kbps
- *Telnet control*: Live DJ operations via port 1234
- *Metadata broadcasting*: Track info sent to listeners

* Network Access

** Local Development
- *Web Interface*: =http://localhost:8080/asteroid/=
- *Live Stream*: =http://localhost:8000/asteroid.mp3=
- *Admin Panel*: =http://localhost:8080/asteroid/admin=

** WSL Network Access
- *WSL IP*: Check with =ip addr show eth0=
- *Web Interface*: =http://[WSL-IP]:8080/asteroid/=
- *Live Stream*: =http://[WSL-IP]:8000/asteroid.mp3=

** Internal Network Broadcasting
- Services bind to all interfaces (0.0.0.0)
- Accessible from any device on local network
- Compatible with media players (VLC, iTunes, etc.)

* Usage Instructions

** Starting the Radio Station
#+BEGIN_SRC bash
# Start Docker streaming services
cd docker
docker-compose up -d

# Start Asteroid web application
sbcl --load asteroid.lisp
#+END_SRC

** Stopping the Radio Station
#+BEGIN_SRC bash
# Stop Docker services
cd docker
docker-compose down
#+END_SRC

** Adding Music
1. Copy MP3/FLAC files to =music/incoming/=
2. Visit admin panel: =http://[IP]:8080/asteroid/admin=
3. Click "Copy Files from Incoming"
4. Files are processed and added to streaming playlist

** Listening to the Stream
- *Web Browser*: Visit main page for embedded player
- *Media Player*: Open =http://[IP]:8000/asteroid.mp3=
- *Mobile Apps*: Use internet radio apps with stream URL

* API Endpoints

** Track Management
- =GET /api/asteroid/tracks= - List all tracks with pagination
- =GET /api/asteroid/tracks/:id/stream= - Stream individual track
- =POST /api/asteroid/scan-library= - Scan and update music library
- =GET /api/asteroid/tracks?search={query}= - Search tracks

** Stream Queue Control (Admin Only)
- =GET /api/asteroid/stream/queue= - Get current stream queue
- =POST /api/asteroid/stream/queue/add= - Add track to queue (position: end/next)
- =POST /api/asteroid/stream/queue/remove= - Remove track from queue
- =POST /api/asteroid/stream/queue/clear= - Clear entire queue
- =POST /api/asteroid/stream/queue/add-playlist= - Add playlist to queue
- =POST /api/asteroid/stream/queue/reorder= - Reorder queue tracks
- =GET /api/asteroid/stream/history= - Get stream history

** User Management
- =POST /api/asteroid/auth/register= - Register new user
- =POST /api/asteroid/auth/login= - User login
- =POST /api/asteroid/auth/logout= - User logout
- =GET /api/asteroid/auth/profile= - Get user profile
- =GET /api/asteroid/users= - List users (admin only)
- =POST /api/asteroid/users/:id/role= - Update user role (admin only)

** Playlist Management
- =GET /api/asteroid/playlists= - List user's playlists
- =POST /api/asteroid/playlists= - Create new playlist
- =GET /api/asteroid/playlists/:id= - Get playlist details
- =DELETE /api/asteroid/playlists/:id= - Delete playlist
- =POST /api/asteroid/playlists/:id/tracks= - Add track to playlist
- =DELETE /api/asteroid/playlists/:id/tracks/:track-id= - Remove track from playlist

See =docs/API-REFERENCE.org= for complete API documentation.

* Database Schema

** Tracks Table
#+BEGIN_SRC lisp
(db:create "tracks" '((title :text)
                      (artist :text)
                      (album :text)
                      (duration :integer)
                      (file-path :text)
                      (format :text)
                      (bitrate :integer)
                      (added-date :integer)
                      (play-count :integer)))
#+END_SRC

** Users Table
#+BEGIN_SRC lisp
(db:create "users" '((username :text)
                     (email :text)
                     (password-hash :text)
                     (role :text)  ; "admin" or "user"
                     (created-at :integer)
                     (last-login :integer)))
#+END_SRC

** Playlists Table
#+BEGIN_SRC lisp
(db:create "playlists" '((name :text)
                         (description :text)
                         (user-id :integer)
                         (created-at :integer)
                         (track-ids :text)))  ; JSON array of track IDs
#+END_SRC

* Dependencies

** Lisp Dependencies (asteroid.asd)
- =:radiance= - Web framework
- =:r-clip= - Templating system
- =:lass= - CSS generation
- =:cl-json= - JSON handling
- =:alexandria= - Utilities
- =:local-time= - Time handling

** System Dependencies
- =icecast2= - Streaming server
- =liquidsoap= - Audio processing
- =taglib= - Metadata extraction (via audio-streams)

* Development Notes

** RADIANCE Configuration
- Domain: "asteroid"
- Routes use =#@= syntax for URL patterns
- Database abstraction via =db:= functions
- CLIP templates with =data-text= attributes

** Database Queries
- Use quoted symbols for field names: =(:= '_id id)=
- RADIANCE returns hash tables with string keys
- Primary key is "_id" internally, "id" in JSON responses

** Streaming Considerations
- MP3 files with spaces in names require playlist.m3u approach
- Liquidsoap fallback prevents stream silence
- Icecast2 mount points must match Liquidsoap configuration

* Future Enhancements

** Planned Features
- Web UI for drag-and-drop queue management
- Real-time now-playing display with WebSocket updates
- Direct browser file uploads with progress bars
- Listener statistics and analytics dashboard
- Scheduled programming and automation
- Social features (playlist sharing, discovery)
- Mobile native applications

** Technical Improvements
- WebSocket integration for real-time updates
- Telnet integration for skip/next commands from web UI
- Auto-queue filling (add tracks when queue runs low)
- Genre-based smart queues
- Listener request system
- Full-text search capabilities

* Troubleshooting

** Common Issues
- *No audio in stream*: Check Liquidsoap logs, verify MP3 files
- *Database errors*: Ensure proper field name quoting in queries
- *Network access*: Verify WSL IP and firewall settings
- *File upload issues*: Check permissions on music directories

** Debugging
- Enable Liquidsoap debug logging: =settings.log.level := 4=
- Check Icecast admin interface for stream status
- Monitor RADIANCE logs for web server issues
- Verify database connectivity and collections

* License

This implementation maintains compatibility with the original Asteroid Radio project license while adding comprehensive streaming capabilities for internet radio broadcasting.