glenn
/
asteroid
mirror of https://github.com/glenneth1/asteroid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
asteroid/docs/PARENSCRIPT-EXPERIMENT.org

14 KiB
Raw Permalink Blame History

ParenScript Conversion Experiment

Overview

This branch experiments with converting all JavaScript files to ParenScript, allowing us to write client-side code in Common Lisp that compiles to JavaScript.

Goals

  • Replace all .js files with ParenScript equivalents
  • Maintain same functionality
  • Improve code maintainability by using one language (Lisp) for both frontend and backend
  • Take advantage of Lisp macros for client-side code generation

Current JavaScript Files

  • static/js/admin.js - Admin dashboard functionality
  • static/js/auth-ui.js - Authentication UI
  • static/js/front-page.js - Front page interactions
  • static/js/player.js - Audio player controls
  • static/js/profile.js - User profile page
  • static/js/users.js - User management

Implementation Plan

Phase 1: Setup [DONE]

  • Add ParenScript dependency to asteroid.asd
  • Create parenscript-utils.lisp with helper functions
  • Create experimental branch

Phase 2: Convert Simple Files First

  • Convert auth-ui.js (smallest, simplest) - COMPLETE ✅
  • Convert front-page.js (stream quality, now playing, pop-out, frameset) - COMPLETE ✅
  • Convert profile.js (user profile, stats, history) - COMPLETE ✅
  • Convert users.js (user management, admin) - COMPLETE ✅

Phase 3: Convert Complex Files

  • Convert player.js (audio player logic) - COMPLETE ✅
  • Convert admin.js (queue management, track controls) - COMPLETE ✅

Phase 4: Testing & Refinement

  • Test all functionality
  • Optimize generated JavaScript
  • Document ParenScript patterns used

Benefits

Code Reuse

  • Share utility functions between frontend and backend
  • Use same data structures and validation logic

Macros

  • Create domain-specific macros for common UI patterns
  • Generate repetitive JavaScript code programmatically

Type Safety

  • Catch more errors at compile time
  • Better IDE support with Lisp tooling

Maintainability

  • Single language for entire stack
  • Easier refactoring across frontend/backend boundary

ParenScript Resources

Lessons Learned

auth-ui.js Conversion (2025-11-06)

Challenge 1: Route Precedence

Problem: Radiance routes are matched in load order, not definition order. The general static file route (/static/(.*)) was intercepting our specific ParenScript route.

Solution: Intercept the static file route and check if path is js/auth-ui.js. If yes, serve ParenScript; otherwise serve regular file.

Challenge 2: Async/Await Syntax

Problem: ParenScript doesn't support async/await syntax. Using (async lambda ...) generated invalid JavaScript.

Solution: Use promise chains with .then() instead of async/await.

Challenge 3: Compile Time vs Runtime

Problem: ParenScript compiler (ps:ps*) isn't available in saved binary at runtime.

Solution: Compile JavaScript at load time and store in a parameter. The function just returns the pre-compiled string.

Success Metrics

  • JavaScript compiles correctly (1386 characters)
  • No browser console errors
  • Auth UI works perfectly (show/hide elements based on login status)
  • Generated code is readable and maintainable

Key Patterns

Compile at load time:

(defparameter *my-js*
  (ps:ps* '(progn ...)))

(defun generate-my-js ()
  *my-js*)

Promise chains instead of async/await:

(ps:chain (fetch url)
          (then (lambda (response) (ps:chain response (json))))
          (then (lambda (data) (process data)))
          (catch (lambda (error) (handle error))))

Intercept static route:

(define-page static #@"/static/(.*)" (:uri-groups (path))
  (if (string= path "js/my-file.js")
      (serve-parenscript)
      (serve-static-file)))

Notes

This is an EXPERIMENTAL branch. The goal is to evaluate ParenScript for this project, not to immediately replace all JavaScript.

If successful, we can merge incrementally, one file at a time.

Conversion Progress

  • auth-ui.js (2025-11-06): Successfully converted. 1386 chars. All functionality working.
  • front-page.js (2025-11-06): Successfully converted. 6900 chars. Stream quality, now playing, pop-out player, frameset mode all working.
  • profile.js (2025-11-06): Successfully converted. Profile data, listening stats, recent tracks, top artists, password change all working.
  • users.js (2025-11-06): Successfully converted. User stats, user list, role changes, activate/deactivate, create user all working.

front-page.js Conversion Notes

This was a more complex file with multiple features. Key learnings:

Global Variables

ParenScript uses (defvar *variable-name* value) for global variables:

(defvar *popout-window* nil)

String Concatenation

Use + operator for string concatenation:

(+ "width=" width ",height=" height)

Conditional Logic

Use cond for multiple conditions in route interception:

(cond
  ((string= path "js/auth-ui.js") ...)
  ((string= path "js/front-page.js") ...)
  (t ...))

Object Property Access

Use ps:getprop for dynamic property access:

(ps:getprop config encoding)  ; config[encoding]

All features tested and working:

  • Stream quality selector changes stream correctly
  • Now playing updates every 10 seconds
  • Pop-out player functionality works
  • Frameset mode toggle works
  • Auto-reconnect on stream errors works

profile.js and users.js Conversion Notes

Modulo Operator

ParenScript doesn't support % for modulo. Use rem (remainder) instead:

;; WRONG:
(% seconds 3600)

;; CORRECT:
(rem seconds 3600)

Property Access with Hyphens

For properties with hyphens (like "last-login"), use ps:getprop:

(ps:getprop user "last-login")
;; Instead of (ps:@ user last-login)

Template Literals in HTML Generation

Build HTML strings with + concatenation:

(+ "<td>" (ps:@ user username) "</td>"
   "<td>" (ps:@ user email) "</td>")

Conditional Attributes

Use if expressions inline for conditional HTML attributes:

(+ "<option value=\"listener\" "
   (if (= (ps:@ user role) "listener") "selected" "")
   ">Listener</option>")

player.js Conversion Notes (2025-11-07)

This was the most challenging conversion due to complex ParenScript compilation errors and server-side error handling issues.

Challenge 1: PUSH Macro Conflict

Problem: Using (push item array) in ParenScript context caused "Error while parsing arguments to DEFMACRO PUSH" because ParenScript doesn't have a PUSH macro like Common Lisp.

Solution: Use array index assignment instead:

;; WRONG:
(push item *play-queue*)

;; CORRECT (what I implemented):
(setf (aref *play-queue* (ps:@ *play-queue* length)) item)

;; ALTERNATIVE (more idiomatic, could be used instead):
(ps:chain *play-queue* (push item))
;; This compiles to: playQueue.push(item);

Note: According to the ParenScript reference manual (/home/glenn/Projects/Code/parenscript/docs/reference.html, lines 672, 745-750), the CHAIN macro is designed to chain together accessors and function calls. This means (ps:chain array (push item)) is actually valid ParenScript and would call the JavaScript push method. Our current implementation using setf and aref works correctly but is more verbose. The chain approach would be more idiomatic JavaScript.

Challenge 2: != Operator

Problem: ParenScript translates !== to a function called bangequals which doesn't exist, causing "bangequals is not defined" runtime error.

Solution: Use (not (= …))= instead:

;; WRONG:
(!= value expected)

;; CORRECT:
(not (== value expected))

Challenge 3: Error Variable Names in handler-case

Problem: ANY variable name used in error handler clauses (e, err, connection-err, condition-object) was being interpreted as an undefined function call, causing errors like "The function ASTEROID::ERR is undefined" or "The function COMMON-LISP:CONDITION is undefined".

Root Cause: When error variables were used in format statements within handler-case error handlers, something in the error handling chain was trying to evaluate them as function calls instead of variables.

Solution: Remove error variable bindings entirely and don't try to print the error object:

;; WRONG:
(handler-case
  (risky-operation)
  (error (err)
    (format t "Error: ~a~%" err)  ; err gets evaluated as function!
    nil))

;; CORRECT:
(handler-case
  (risky-operation)
  (error ()
    (format t "Error occurred~%")  ; No variable to evaluate
    nil))

Challenge 4: Parenthesis Imbalance in handler-case

Problem: Using (condition (var) ...) as error handler type caused "end of file" errors because condition is not a valid error type in handler-case, and t is also invalid.

Solution: Use error as the catch-all error type:

;; WRONG:
(handler-case
  (risky-operation)
  (t () ...))           ; t is not valid
  (condition () ...))   ; condition needs to be a type

;; CORRECT:
(handler-case
  (risky-operation)
  (error ()             ; error is the correct catch-all type
    (format t "Error occurred~%")
    nil))

Challenge 5: let* Structure with handler-case

Problem: When adding handler-case with a progn wrapper, the let* binding was closed before the when block that used its variables, causing "end of file" errors.

Solution: Keep let* as the main form and put all logic inside it:

;; WRONG:
(handler-case
  (progn
    (let* ((url "...")
           (response (fetch url)))
      (when response ...)))  ; let* closed too early!
  (error () nil))

;; CORRECT:
(let* ((url "...")
       (response (fetch url)))
  (when response
    ...))  ; All logic inside let*

Challenge 6: Icecast Listener Count Aggregation

Problem: Function only checked /asteroid.mp3 mount point, missing listeners on /asteroid.aac and /asteroid-low.mp3 streams.

Solution: Modified icecast-now-playing function in frontend-partials.lisp to loop through all three mount points and aggregate listener counts:

(let ((total-listeners 0))
  (dolist (mount '("/asteroid\\.mp3" "/asteroid\\.aac" "/asteroid-low\\.mp3"))
    (let ((match-pos (cl-ppcre:scan (format nil "<source mount=\"~a\">" mount) xml-string)))
      (when match-pos
        (let* ((source-section (subseq xml-string match-pos ...))
               (listenersp (cl-ppcre:all-matches "<listeners>" source-section)))
          (when listenersp
            (let ((count (parse-integer (cl-ppcre:regex-replace-all 
                                         ".*<listeners>(.*?)</listeners>.*" 
                                         source-section "\\1") 
                                        :junk-allowed t)))
              (incf total-listeners count)))))))
  total-listeners)

Additional Changes to frontend-partials.lisp:

  • Fixed stray ^ character in (in-package :asteroid) form
  • Added error handler to define-api asteroid/partial/now-playing endpoint to catch errors gracefully
  • Added debug logging to track Icecast stats fetching and parsing
  • Removed problematic error variable usage in error handlers (see Challenge 3)

Success Metrics

  • player.lisp compiles without errors
  • All player functionality works (play, pause, queue, playlists)
  • Now Playing section displays correctly with live track information
  • Listener count aggregates across all three streams
  • No JavaScript runtime errors in browser console
  • No server-side Lisp errors

Summary of Key ParenScript Patterns

  1. Async/Await: Use promise chains with .then() instead
  2. Modulo: Use rem instead of %
  3. Global Variables: Use defvar with asterisks: *variable-name*
  4. String Concatenation: Use + operator
  5. Property Access: Use ps:getprop for dynamic/hyphenated properties
  6. Object Creation: Use ps:create with keyword arguments
  7. Array Methods: Use ps:chain for method chaining
  8. Route Interception: Use cond in static route handler
  9. Compile at Load Time: Store compiled JS in defparameter
  10. Return Pre-compiled String: Function just returns the parameter value
  11. Array Push: Use (setf (aref array (ps:@ array length)) item) instead of push
  12. Not Equal: Use (not (= …))= instead of !=
  13. Error Handlers: Don't use error variable names in format statements; use error type for catch-all
  14. Parenthesis Balance: Keep let* as main form, don't wrap with progn inside handler-case

Final Status (2025-11-07)

✅ ALL JAVASCRIPT FILES SUCCESSFULLY CONVERTED TO PARENSCRIPT

The ParenScript migration is complete! All client-side JavaScript is now generated from Common Lisp code. The application maintains 100% of its original functionality while using a single language (Lisp) for both frontend and backend.

Files converted:

  • auth-ui.js → parenscript/auth-ui.lisp
  • front-page.js → parenscript/front-page.lisp
  • profile.js → parenscript/profile.lisp
  • users.js → parenscript/users.lisp
  • player.js → parenscript/player.lisp
  • admin.js → parenscript/admin.lisp

The experiment was a success. We can now maintain the entire Asteroid Radio codebase in Common Lisp.