asteroid/project-summary.org

#+TITLE: Asteroid Radio Web Server Implementation Summary
#+DATE: 2025-08-20
#+AUTHOR: Development Session Summary

* Project Overview

This document summarizes the implementation of a basic web server for the Asteroid Radio project, a Common Lisp-based streaming radio station for "asteroid music for hackers."

** Project Context
- Fork of https://github.com/fade/asteroid
- Goal: Create streaming radio station with web interface
- Started with basic Common Lisp framework skeleton
- Implemented web server as foundation for future streaming components

* Initial State Analysis

** Original Dependencies (asteroid.asd)
- :RADIANCE (web framework)
- :MITO (database ORM)
- :MITO-AUTH (authentication)
- :STR (string utilities)
- :PZMQ (ZeroMQ bindings)
- :SPINNERET (HTML generation)

** Original Code Structure
- =asteroid.lisp= - Main package with placeholder =-main= function
- =app-utils.lisp= - Utility functions for debugger control and cross-platform quit
- =design.org= - Comprehensive feature specification and MVP roadmap

* Dependency Changes and Rationale

** Removed Dependencies
| Dependency | Reason for Removal |
|------------|-------------------|
| :RADIANCE | Requires separate Shirakumo dist installation, switched to simpler Hunchentoot for MVP |
| :MITO | Not available in default Quicklisp, not needed for basic web server |
| :MITO-AUTH | Not available in default Quicklisp, authentication not needed for MVP |
| :STR | Not used in current implementation |
| :PZMQ | Not needed for basic web server functionality |

** Final Dependencies
| Dependency | Purpose | Justification |
|------------|---------|---------------|
| :RADIANCE | Web framework | Modular web framework with subdomain routing and integrated module system |
| :SPINNERET | HTML generation | Clean DSL for generating HTML, integrates well with Common Lisp |
| :CL-JSON | JSON encoding/decoding | Standard library for API endpoints |

** Migration from Hunchentoot to RADIANCE
- **Initial Choice**: Started with Hunchentoot for simpler MVP setup
- **Discovery**: RADIANCE available via Shirakumo dist: `(ql-dist:install-dist "http://dist.shirakumo.org/shirakumo.txt")`
- **Migration Completed**: Successfully migrated to RADIANCE framework
- **Benefits**: RADIANCE provides modular architecture, subdomain routing, and integrated module system
- **Result**: More scalable foundation for future radio station features

* Implementation Details

** Web Server Architecture
- Framework: RADIANCE modular web framework
- Port: 8080 (default RADIANCE configuration)
- Module: asteroid (domain: "asteroid")
- Subdomain routing: asteroid.localhost:8080
- Server management: =radiance:startup= / =radiance:shutdown=

** Route Structure (RADIANCE)
| Route | Handler | Purpose | URL |
|-------|---------|---------|-----|
| / | =index= | Main page with station status | http://asteroid.localhost:8080/ |
| /admin | =admin= | Admin dashboard with controls | http://asteroid.localhost:8080/admin |
| /player | =player= | Web player interface | http://asteroid.localhost:8080/player |
| /api/status | =api/status= | JSON API endpoint | http://asteroid.localhost:8080/api/status |

** HTML Generation Strategy
- Direct use of =spinneret:with-html-string= in each handler
- Consistent hacker-themed styling (green text, black background)
- Responsive design with CSS embedded in each page

* Errors Encountered and Solutions

** Error 1: Missing MITO Dependency
*** Problem
#+BEGIN_EXAMPLE
debugger invoked on a ASDF/FIND-COMPONENT:MISSING-DEPENDENCY
Component :MITO not found, required by #<SYSTEM "asteroid">
#+END_EXAMPLE

*** Root Cause
MITO and related database dependencies not available in Quicklisp distribution.

*** Solution
Removed unused dependencies from =asteroid.asd=:
- Removed :MITO, :MITO-AUTH, :STR, :PZMQ
- Kept only essential dependencies: :HUNCHENTOOT, :SPINNERET, :CL-JSON

** Error 2: RADIANCE Framework Migration
*** Problem
Initially avoided RADIANCE due to perceived unavailability, implemented with Hunchentoot instead.

*** Root Cause
RADIANCE available via Shirakumo dist: `(ql-dist:install-dist "http://dist.shirakumo.org/shirakumo.txt")` but required additional setup step.

*** Solution
- Successfully migrated from Hunchentoot to RADIANCE
- Installed Shirakumo distribution for RADIANCE access
- Rewrote route handlers using =define-page= syntax
- Added =define-module= declaration for proper RADIANCE integration
- Updated server management to use =radiance:startup= / =radiance:shutdown=

** Error 3: HTML Generation Function Signature Mismatch
*** Problem
#+BEGIN_EXAMPLE
The function GENERATE-PAGE-HTML is called with five arguments, but wants exactly two.
#+END_EXAMPLE

*** Root Cause
Initial =generate-page-html= helper function designed for single body argument, but called with multiple arguments.

*** Solution
Attempted fix with =&rest= parameter, but Spinneret macro expansion issues persisted.

** Error 4: RADIANCE Route Syntax Issues
*** Problem
#+BEGIN_EXAMPLE
Module #<PACKAGE "ASTEROID"> requested but while the package exists, it is not a module.
The value #@"asteroid/api/status" is not of type LIST
#+END_EXAMPLE

*** Root Cause
- Missing =define-module= declaration for RADIANCE integration
- Incorrect route path syntax using =asteroid/= prefix instead of module-relative paths
- Wrong API endpoint definition syntax

*** Solution
- Added =define-module= declaration with proper domain specification
- Fixed route paths: =#@"asteroid/"= → =#@"/"=, =#@"asteroid/admin"= → =#@"/admin"=
- Updated API endpoint to use =define-page= instead of =define-api=
- Fixed parentheses syntax errors in HTML generation

** Error 5: Shell History Expansion Issues
*** Problem
#+BEGIN_EXAMPLE
zsh: event not found: ~
zsh: event not found: \
#+END_EXAMPLE

*** Root Cause
Zsh history expansion interfering with command-line arguments containing special characters.

*** Solution
Used single quotes instead of double quotes for SBCL command-line arguments to prevent shell interpretation.

* Current Project Status

** ✅ Completed Features
- [X] Basic web server running on localhost:8080
- [X] Main page with station status display
- [X] Admin dashboard with placeholder controls
- [X] Web player interface (UI only)
- [X] JSON API endpoint (/api/status)
- [X] Hacker-themed consistent styling
- [X] Proper error handling and server management
- [X] Git upstream remote configuration

** 🎯 Current Capabilities
- Web server starts/stops cleanly
- All routes functional and accessible
- HTML generation working correctly
- JSON API returning structured data
- Responsive web interface
- Server management functions exported

** 🚀 Running the Server

*** RADIANCE Setup (One-time)
#+BEGIN_EXAMPLE
sbcl --eval '(ql-dist:install-dist "http://dist.shirakumo.org/shirakumo.txt")'
#+END_EXAMPLE

*** Command Line (One-shot execution)
#+BEGIN_EXAMPLE
sbcl --eval '(ql:quickload (quote (:radiance :spinneret :cl-json)))' \
     --eval '(load "asteroid.asd")' \
     --eval '(asdf:load-system :asteroid)' \
     --eval '(asteroid:start-server)' \
     --eval '(format t "Server running at http://asteroid.localhost:8080/ - Press Ctrl+C to stop")'
#+END_EXAMPLE

*** Interactive REPL
#+BEGIN_EXAMPLE
sbcl
(ql:quickload '(:radiance :spinneret :cl-json))
(load "asteroid.asd")
(asdf:load-system :asteroid)
(asteroid:start-server)
;; Server now running at http://asteroid.localhost:8080/
;; To stop: (asteroid:stop-server)
#+END_EXAMPLE

*** Available Functions
- =(asteroid:start-server)= - Start RADIANCE server (non-blocking)
- =(asteroid:stop-server)= - Stop RADIANCE server cleanly
- =(asteroid:run-server)= - Start server and keep running (blocking, with Ctrl+C handler)

*** Access URLs
- **Main page**: http://asteroid.localhost:8080/
- **Admin dashboard**: http://asteroid.localhost:8080/admin
- **Web player**: http://asteroid.localhost:8080/player
- **API endpoint**: http://asteroid.localhost:8080/api/status
- **RADIANCE welcome**: http://localhost:8080/

** 📋 Next Steps (Not Implemented)
- Database integration (when MITO alternative chosen)
- Audio streaming backend (Liquidsoap integration)
- Icecast server integration
- File upload functionality
- Authentication system
- Real-time now-playing updates
- WebSocket integration for live updates
- **Completed**: ✅ Successfully migrated to RADIANCE framework

* Technical Lessons Learned

** Dependency Management
- Always verify dependency availability in target package manager
- Prefer widely-adopted, stable libraries over newer alternatives
- Keep dependency list minimal for initial implementation

** Common Lisp Web Development
- RADIANCE provides modular architecture with subdomain routing
- Spinneret works best with direct macro usage, not through helper functions
- HTML generation should be kept simple and direct
- RADIANCE modules require proper =define-module= declarations
- Route paths in RADIANCE are module-relative (use =#@"/"= not =#@"asteroid/"=)

** Error Handling Strategy
- Compilation warnings often indicate runtime issues
- Test each component incrementally
- Use REPL for interactive debugging and testing

** Development Workflow
- Start with minimal working version
- Add complexity incrementally
- Test each change immediately
- Keep fallback options for critical dependencies

* File Structure Summary

#+BEGIN_EXAMPLE
asteroid/
├── asteroid.asd          # System definition (minimal dependencies)
├── asteroid.lisp         # Main web server implementation
├── app-utils.lisp        # Utility functions
├── design.org           # Original project specification
├── test-server.lisp     # Server testing script
├── project-summary.org  # This document
├── Makefile            # Build configuration
└── LICENSE             # AGPL v3 license
#+END_EXAMPLE

* Conclusion

Successfully implemented and migrated a functional web server foundation for the Asteroid Radio project using RADIANCE framework. The server provides a complete web interface with admin controls, player interface, and API endpoints accessible via subdomain routing at asteroid.localhost:8080.

Key achievements:
- **Framework Migration**: Successfully migrated from Hunchentoot to RADIANCE
- **Modular Architecture**: Implemented proper RADIANCE module with subdomain routing
- **Complete Web Interface**: Main page, admin dashboard, web player, and JSON API
- **Scalable Foundation**: RADIANCE provides better architecture for future radio features

The implementation demonstrates the value of exploring framework alternatives and provides a robust, modular foundation ready for the next development phase: integrating audio streaming components (Liquidsoap/Icecast) and database functionality.