Go to file
glenneth 0dee51f344 Update DEV-LOG.org with comprehensive development history 2025-10-17 05:02:15 +03:00
config Complete Templates section: CLIP refactoring, user management, pagination, playlists, UI fixes, PostgreSQL setup 2025-10-04 09:54:04 -04:00
data/sessions feat: Add auto-scan on startup and live Icecast/Liquidsoap status checks 2025-10-04 09:54:04 -04:00
docker Improve audio quality and streaming performance 2025-10-15 06:38:53 -04:00
docs Update DEV-LOG.org with comprehensive development history 2025-10-17 05:02:15 +03:00
music Small moves. delete some binary data and add users.lisp 2025-09-30 12:50:45 -04:00
static fix: playlist create button wrap on small screens 2025-10-15 06:40:20 -04:00
template Add admin UI for stream queue management 2025-10-15 06:38:53 -04:00
.gitignore feat: Add auto-scan on startup and live Icecast/Liquidsoap status checks 2025-10-04 09:54:04 -04:00
AAC-STREAMING.org docs: Convert AAC-STREAMING.md to org-mode format 2025-10-02 16:51:59 +03:00
LICENSE Initial commit 2025-08-12 10:42:34 -04:00
Makefile clean out the Makefile a bit. 2025-09-04 11:58:41 -04:00
README.org docs: Comprehensive documentation update for current features 2025-10-16 13:41:08 +03:00
SESSION-NOTES-2025-10-12.org docs: Add session notes for page flow feature implementation 2025-10-12 09:47:38 -04:00
TODO.org docs: Mark Page Flow feature as complete in TODO 2025-10-12 09:47:38 -04:00
analyze-performance.py feat: Add auto-scan on startup and live Icecast/Liquidsoap status checks 2025-10-04 09:54:04 -04:00
app-utils.lisp Project skeleton. 2025-08-12 10:59:34 -04:00
asteroid-radio.liq Implement complete internet radio streaming system 2025-09-11 15:30:01 +03:00
asteroid.asd Add stream queue control system 2025-10-15 06:38:53 -04:00
asteroid.lisp fix: avoid icecast xml to be shown on frontend when there is no artist 2025-10-15 06:41:07 -04:00
auth-routes.lisp feat: Implement role-based page flow and user management APIs 2025-10-12 09:47:38 -04:00
build-executable.lisp Fix Asteroid Radio authentication system 2025-09-30 13:20:58 -04:00
comprehensive-performance-test.sh feat: Add auto-scan on startup and live Icecast/Liquidsoap status checks 2025-10-04 09:54:04 -04:00
database.lisp Fix Asteroid Radio authentication system 2025-09-30 13:20:58 -04:00
design.org Add detailed design document for Asteroid Music radio station (#1) 2025-08-12 11:32:39 -04:00
module.lisp tiny cleanup. 2025-09-08 11:07:02 -04:00
playlist-management.lisp Fix playlist schema mismatch - use track-ids field consistently 2025-10-07 18:39:49 -04:00
playlist.m3u Implement complete internet radio streaming system 2025-09-11 15:30:01 +03:00
project-summary.org Migrate from Hunchentoot to RADIANCE framework 2025-08-20 09:40:14 -04:00
run-all-tests.sh feat: Add auto-scan on startup and live Icecast/Liquidsoap status checks 2025-10-04 09:54:04 -04:00
setup-environment.lisp feat: Add auto-scan on startup and live Icecast/Liquidsoap status checks 2025-10-04 09:54:04 -04:00
simple-analysis.py feat: Add auto-scan on startup and live Icecast/Liquidsoap status checks 2025-10-04 09:54:04 -04:00
stream-control.lisp Add stream queue control system 2025-10-15 06:38:53 -04:00
stream-media.lisp Add stream queue control system 2025-10-15 06:38:53 -04:00
stream-queue.m3u Add stream queue control system 2025-10-15 06:38:53 -04:00
template-utils.lisp Complete CLIP template refactoring and all template features 2025-10-04 09:54:04 -04:00
test-server.sh Add comprehensive automated test suite 2025-10-08 22:56:54 -04:00
test-user-api.sh Complete CLIP template refactoring and all template features 2025-10-04 09:54:04 -04:00
user-management.lisp feat: Implement role-based page flow and user management APIs 2025-10-12 09:47:38 -04:00
users.lisp feat: Add Docker streaming infrastructure for Liquidsoap and Icecast2 2025-10-02 16:50:06 +03:00

README.org

Asteroid Radio - Internet Streaming Platform

Overview

Asteroid Radio is a modern, web-based music streaming platform built with Common Lisp and the Radiance framework. It provides a complete internet radio streaming system with live broadcasting, user management, playlist control, and professional audio processing.

Key Features

Live Internet Radio Streaming

  • Multiple streaming formats: AAC 96kbps (high efficiency), MP3 128kbps (standard), MP3 64kbps (low bandwidth)
  • Professional audio processing with ReplayGain for consistent volume without pumping
  • Smooth crossfading between tracks (5 second transitions)
  • Dynamic compression to prevent clipping
  • Icecast2 streaming server integration
  • Liquidsoap audio pipeline with watch-based playlist reloading

Stream Queue Control System

  • Curated stream queue management (play tracks in specific order)
  • Add tracks to end of queue or as "next to play"
  • Add entire playlists to stream queue
  • Real-time queue reordering
  • Automatic fallback to random playback when queue is empty
  • Stream history tracking
  • Admin-only queue control via REST API

Music Library Management

  • PostgreSQL-backed track storage with metadata extraction
  • Support for MP3, FLAC, OGG, and WAV formats
  • Automatic metadata extraction using taglib
  • Track search, filtering, and sorting capabilities
  • Pagination support for large libraries
  • Individual track streaming on-demand

User Management System

  • User registration and authentication
  • Role-based access control (admin/user)
  • User profiles and session management
  • Personal playlist creation and management
  • Admin interface for user administration

Web Interface

  • RADIANCE framework with CLIP templating
  • Modern admin dashboard with JavaScript controls
  • Web player with HTML5 audio controls
  • Live stream integration with embedded player
  • Dark hacker-themed aesthetic with VT323 font
  • Responsive design for desktop and mobile

Network Broadcasting

  • Docker-based streaming infrastructure
  • WSL-compatible networking for internal network access
  • Professional streaming URLs for media players
  • Multi-listener support via Icecast2
  • Telnet control interface for live DJ operations

Architecture Changes

Framework Migration

  • Migrated from Hunchentoot to RADIANCE web framework
  • Implemented proper domain routing (/asteroid/)
  • CLIP templating system for dynamic content
  • Database abstraction layer for track storage

Streaming Stack

  • Icecast2: Streaming server (port 8000)
  • Liquidsoap: Audio processing and streaming pipeline with telnet control (port 1234)
  • RADIANCE: Web server and API (port 8080)
  • PostgreSQL: User accounts, track metadata, and playlist storage
  • Docker Compose: Container orchestration for streaming services

File Structure

asteroid/
├── asteroid.lisp              # Main server with RADIANCE routes
├── asteroid.asd               # System definition with dependencies
├── stream-control.lisp        # Stream queue management system
├── stream-media.lisp          # Media streaming and track management
├── user-management.lisp       # User authentication and profiles
├── playlist-management.lisp   # User playlist operations
├── auth-routes.lisp           # Authentication endpoints
├── stream-queue.m3u           # Generated stream queue playlist
├── docker/                    # Docker streaming infrastructure
│   ├── docker-compose.yml     # Container orchestration
│   ├── asteroid-radio-docker.liq  # Liquidsoap configuration
│   └── start.sh               # Container startup script
├── template/                  # CLIP HTML templates
│   ├── front-page.chtml       # Main page with live stream
│   ├── admin.chtml            # Admin dashboard with queue controls
│   ├── player.chtml           # Web player interface
│   └── login.chtml            # User authentication
├── static/                    # Frontend assets
│   ├── asteroid.lass          # LASS stylesheet source
│   ├── asteroid.css           # Compiled CSS
│   └── js/
│       ├── admin.js           # Admin interface controls
│       └── player.js          # Web player functionality
├── docs/                      # Documentation
│   ├── STREAM-CONTROL.org     # Queue management guide
│   ├── API-REFERENCE.org      # Complete API documentation
│   ├── USER-MANAGEMENT-SYSTEM.org  # User system guide
│   └── INSTALLATION.org       # Setup instructions
└── music/                     # Music library
    └── library/               # Music files

Track Upload Workflow

Current Implementation (Manual Upload)

  1. Copy files to staging: Place MP3/FLAC files in music/incoming/
  2. Access admin panel: Navigate to http://[IP]:8080/asteroid/admin
  3. Process files: Click "Copy Files from Incoming" button
  4. Database update: Files are moved to music/library/ and metadata extracted
  5. Automatic playlist: playlist.m3u is regenerated for streaming

File Processing Steps

  1. Files copied from music/incoming/ to music/library/
  2. Metadata extracted using taglib (title, artist, album, duration, bitrate)
  3. Database record created with file path and metadata
  4. Playlist file updated for Liquidsoap streaming
  5. Files immediately available for on-demand streaming

Supported Formats

  • MP3: Primary format, best compatibility
  • FLAC: Lossless audio, high quality
  • OGG: Open source format
  • WAV: Uncompressed audio

Icecast2 Integration

Configuration

  • Server: localhost:8000
  • Mount point: /asteroid.mp3
  • Password: b3l0wz3r0 (configured in Liquidsoap)
  • Format: MP3 128kbps stereo

Installation (Ubuntu/Debian)

sudo apt update
sudo apt install icecast2
sudo systemctl enable icecast2
sudo systemctl start icecast2

Stream Access

  • Direct URL: http://[IP]:8000/asteroid.mp3
  • Admin interface: http://[IP]:8000/admin/
  • Statistics: http://[IP]:8000/status.xsl

Liquidsoap Integration

Configuration File: docker/asteroid-radio-docker.liq

#!/usr/bin/liquidsoap

# Allow running as root in Docker
set("init.allow_root", true)
log.level.set(4)

# Enable telnet server for remote control
settings.server.telnet.set(true)
settings.server.telnet.port.set(1234)
settings.server.telnet.bind_addr.set("0.0.0.0")

# Create playlist source from managed stream queue
radio = playlist(
  mode="normal",  # Play in order (not randomized)
  reload=5,       # Check for playlist updates every 5 seconds
  reload_mode="watch",  # Watch file for changes
  "/app/stream-queue.m3u"
)

# Fallback to directory scan if queue is empty
radio_fallback = playlist.safe(
  mode="randomize",
  reload=3600,
  "/app/music/"
)

radio = fallback(track_sensitive=false, [radio, radio_fallback])

# Use ReplayGain for consistent volume without pumping
radio = amplify(1.0, override="replaygain", radio)

# Add smooth crossfade between tracks (5 seconds)
radio = crossfade(
  duration=5.0,
  fade_in=3.0,
  fade_out=3.0,
  radio
)

# Add compressor to prevent clipping
radio = compress(
  ratio=3.0,
  threshold=-15.0,
  attack=50.0,
  release=400.0,
  radio
)

# Output to Icecast2 in multiple formats
output.icecast(%mp3(bitrate=128), host="icecast", port=8000,
  password="H1tn31EhsyLrfRmo", mount="asteroid.mp3",
  name="Asteroid Radio", radio)

output.icecast(%fdkaac(bitrate=96), host="icecast", port=8000,
  password="H1tn31EhsyLrfRmo", mount="asteroid.aac",
  name="Asteroid Radio (AAC)", radio)

output.icecast(%mp3(bitrate=64), host="icecast", port=8000,
  password="H1tn31EhsyLrfRmo", mount="asteroid-low.mp3",
  name="Asteroid Radio (Low Quality)", radio)

Installation (Ubuntu/Debian)

sudo apt update
sudo apt install liquidsoap

Features

  • Stream queue control: Curated playlist with watch-based reloading (5 second updates)
  • Smart fallback: Random playback when queue is empty
  • ReplayGain processing: Consistent volume without pumping artifacts
  • Crossfading: Smooth 5-second transitions between tracks
  • Dynamic compression: Prevents clipping and maintains audio quality
  • Multi-format output: AAC 96kbps, MP3 128kbps, MP3 64kbps
  • Telnet control: Live DJ operations via port 1234
  • Metadata broadcasting: Track info sent to listeners

Network Access

Local Development

  • Web Interface: http://localhost:8080/asteroid/
  • Live Stream: http://localhost:8000/asteroid.mp3
  • Admin Panel: http://localhost:8080/asteroid/admin

WSL Network Access

  • WSL IP: Check with ip addr show eth0
  • Web Interface: http://[WSL-IP]:8080/asteroid/
  • Live Stream: http://[WSL-IP]:8000/asteroid.mp3

Internal Network Broadcasting

  • Services bind to all interfaces (0.0.0.0)
  • Accessible from any device on local network
  • Compatible with media players (VLC, iTunes, etc.)

Usage Instructions

Starting the Radio Station

# Start Docker streaming services
cd docker
docker-compose up -d

# Start Asteroid web application
sbcl --load asteroid.lisp

Stopping the Radio Station

# Stop Docker services
cd docker
docker-compose down

Adding Music

  1. Copy MP3/FLAC files to music/incoming/
  2. Visit admin panel: http://[IP]:8080/asteroid/admin
  3. Click "Copy Files from Incoming"
  4. Files are processed and added to streaming playlist

Listening to the Stream

  • Web Browser: Visit main page for embedded player
  • Media Player: Open http://[IP]:8000/asteroid.mp3
  • Mobile Apps: Use internet radio apps with stream URL

API Endpoints

Track Management

  • GET /api/asteroid/tracks - List all tracks with pagination
  • GET /api/asteroid/tracks/:id/stream - Stream individual track
  • POST /api/asteroid/scan-library - Scan and update music library
  • GET /api/asteroid/tracks?search={query} - Search tracks

Stream Queue Control (Admin Only)

  • GET /api/asteroid/stream/queue - Get current stream queue
  • POST /api/asteroid/stream/queue/add - Add track to queue (position: end/next)
  • POST /api/asteroid/stream/queue/remove - Remove track from queue
  • POST /api/asteroid/stream/queue/clear - Clear entire queue
  • POST /api/asteroid/stream/queue/add-playlist - Add playlist to queue
  • POST /api/asteroid/stream/queue/reorder - Reorder queue tracks
  • GET /api/asteroid/stream/history - Get stream history

User Management

  • POST /api/asteroid/auth/register - Register new user
  • POST /api/asteroid/auth/login - User login
  • POST /api/asteroid/auth/logout - User logout
  • GET /api/asteroid/auth/profile - Get user profile
  • GET /api/asteroid/users - List users (admin only)
  • POST /api/asteroid/users/:id/role - Update user role (admin only)

Playlist Management

  • GET /api/asteroid/playlists - List user's playlists
  • POST /api/asteroid/playlists - Create new playlist
  • GET /api/asteroid/playlists/:id - Get playlist details
  • DELETE /api/asteroid/playlists/:id - Delete playlist
  • POST /api/asteroid/playlists/:id/tracks - Add track to playlist
  • DELETE /api/asteroid/playlists/:id/tracks/:track-id - Remove track from playlist

See docs/API-REFERENCE.org for complete API documentation.

Database Schema

Tracks Table

(db:create "tracks" '((title :text)
                      (artist :text)
                      (album :text)
                      (duration :integer)
                      (file-path :text)
                      (format :text)
                      (bitrate :integer)
                      (added-date :integer)
                      (play-count :integer)))

Users Table

(db:create "users" '((username :text)
                     (email :text)
                     (password-hash :text)
                     (role :text)  ; "admin" or "user"
                     (created-at :integer)
                     (last-login :integer)))

Playlists Table

(db:create "playlists" '((name :text)
                         (description :text)
                         (user-id :integer)
                         (created-at :integer)
                         (track-ids :text)))  ; JSON array of track IDs

Dependencies

Lisp Dependencies (asteroid.asd)

  • :radiance - Web framework
  • :r-clip - Templating system
  • :lass - CSS generation
  • :cl-json - JSON handling
  • :alexandria - Utilities
  • :local-time - Time handling

System Dependencies

  • icecast2 - Streaming server
  • liquidsoap - Audio processing
  • taglib - Metadata extraction (via audio-streams)

Development Notes

RADIANCE Configuration

  • Domain: "asteroid"
  • Routes use #@ syntax for URL patterns
  • Database abstraction via db: functions
  • CLIP templates with data-text attributes

Database Queries

  • Use quoted symbols for field names: (: '_id id)=
  • RADIANCE returns hash tables with string keys
  • Primary key is "_id" internally, "id" in JSON responses

Streaming Considerations

  • MP3 files with spaces in names require playlist.m3u approach
  • Liquidsoap fallback prevents stream silence
  • Icecast2 mount points must match Liquidsoap configuration

Future Enhancements

Planned Features

  • Web UI for drag-and-drop queue management
  • Real-time now-playing display with WebSocket updates
  • Direct browser file uploads with progress bars
  • Listener statistics and analytics dashboard
  • Scheduled programming and automation
  • Social features (playlist sharing, discovery)
  • Mobile native applications

Technical Improvements

  • WebSocket integration for real-time updates
  • Telnet integration for skip/next commands from web UI
  • Auto-queue filling (add tracks when queue runs low)
  • Genre-based smart queues
  • Listener request system
  • Full-text search capabilities

Troubleshooting

Common Issues

  • No audio in stream: Check Liquidsoap logs, verify MP3 files
  • Database errors: Ensure proper field name quoting in queries
  • Network access: Verify WSL IP and firewall settings
  • File upload issues: Check permissions on music directories

Debugging

  • Enable Liquidsoap debug logging: settings.log.level : 4=
  • Check Icecast admin interface for stream status
  • Monitor RADIANCE logs for web server issues
  • Verify database connectivity and collections

License

This implementation maintains compatibility with the original Asteroid Radio project license while adding comprehensive streaming capabilities for internet radio broadcasting.