[Feature]: Android app: music player

Open
#46 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
20/100
Issue type
Feature
Clarity
Clearly specified
Activity status
Active
Tech stack
android, kotlin, react-native, rust, typescript

Research direction

Read the epic #40 first, then review #41 and #42 for the Android build plumbing and #47 for the music cache. Trace the web behavior through web/decoder/src/lib.rs, web/src/music/decoder.worker.ts, web/src/music/engine.ts, and web/src/music/player.ts; the issue names the mobile audio module and UI entry points as well. Done means meeting the playback, background controls, shared queue, web regression, and Android Opus acceptance criteria.

Written by the indexing model from the issue text.

Description

enhancement

[!NOTE]
This issue was written by Claude (Claude Opus 5.5) from a planning session with @PhrolovaMoe. Part of #40, builds on #41 and #42. Read the epic first.

Area

Streaming / playback

Problem

The web music player is a custom engine that gives sample-accurate gapless playback, crossfade and ReplayGain:

  • a Rust decoder (web/decoder, Symphonia) compiled to WASM, run in a worker that reads files over synchronous Range XHRs,
  • crossfade/sequencing logic in web/src/music/decoder.worker.ts,
  • a SharedArrayBuffer ring played by an AudioWorklet.

None of that runs in React Native. On Android, music also has to keep playing with the screen off and be controllable from the lock screen, notification shade, Bluetooth and the OEM surfaces (Samsung Now Bar, HyperOS island, OPPO/OnePlus Live Alerts).

Proposed solution

Share the decoding and sequencing in Rust, run it natively behind a Media3 player inside a media service, and keep the queue logic in TypeScript shared with web.

1. Rust: one core, two frontends
  • New crate decoder/core (tinystream-decoder-core, rlib, no wasm deps), added to the Cargo workspace. It contains:
    • What web/decoder/src/lib.rs does today: probing, decoding, trimming encoder delay and padding, resampling with rubato to the output rate, stereo float output, gain, seek. The HTTP reading is abstracted behind a trait, e.g. trait RangeSource { fn len(&self) -> io::Result<u64>; fn read_range(&self, start: u64, end: u64) -> io::Result<Vec<u8>>; }. The block cache (BLOCK 512 KB, KEEP 12) stays in the core.
    • The sequencer now in decoder.worker.ts pump()/begin(): a list of items { key, source, size, ext, gain, crossfade }, decoding one into the next without gaps, crossfade mixing when crossfade > 0, skipping items that fail to open while reporting the error, and producing "mark" events (key, frame, offset, duration) and an "end" event. This makes gapless/crossfade logic exist once.
  • web/decoder becomes a thin wasm-bindgen frontend over the core, with the same JS API as today plus the sequencer. Simplify decoder.worker.ts to: create sources through tsReadRange, feed the sequencer, copy its output into the ring, and forward its events. The web player must behave exactly as before (gapless, crossfade, ReplayGain, error fallback to FLAC, seeking).
  • mobile/modules/audio/rust (tinystream-decoder-android): a uniffi frontend over the core, built by the #41 cargo-ndk plumbing.
    • RangeSource is implemented in Kotlin through a uniffi callback interface, so all network and file I/O goes through OkHttp with the app's network config (cleartext, user CAs, Bearer), and through the music cache (#47) when a file is cached.
  • Codecs stay as today (flac, mp3, aac, alac, isomp4, ogg/vorbis, wav, aiff, pcm). Opus: add an optional opus feature to the core (libopus through a Rust binding, Ogg Opus with pre-skip honoured so it stays gapless). Enable it for Android only, because #47 stores Opus. It's off for wasm, so the web bundle doesn't grow.
2. Native engine (mobile/modules/audio, Kotlin)
  • An output thread pulls from the sequencer into an AudioTrack (float PCM, stereo, at the device's native output rate from AudioManager.PROPERTY_OUTPUT_SAMPLE_RATE, low-latency off, a buffer of a few hundred ms).
  • The position is derived from frames played (AudioTrack.getPlaybackHeadPosition) and the sequencer's marks, the same model as web/src/music/engine.ts.
  • The engine's JS API mirrors the web Engine class, so the queue code can drive either:
    • play(items, start), upcoming(after, items), gain(key, gain), stop(), pause()/resume()
    • events track(key) (audible now, not just decoded), ended, error(key, message), starved(bool), and the position
  • Wrap it in a Media3 SimpleBasePlayer inside a MediaSessionService (foreground service type mediaPlayback):
    • DefaultMediaNotificationProvider with artwork (authenticated cover URL through a custom BitmapLoader), title, artist and album
    • custom commands: like/star, shuffle, repeat
    • seeking from the notification and lock screen
    • this is what makes Now Bar / islands / Live Alerts, the lock screen and Bluetooth/headset buttons work, with no OEM-specific code
  • Media-session commands (next, previous, seek, play/pause from the notification or headset) are sent to JS as events. JS decides, exactly as for in-app taps. The foreground service keeps the process (and JS) alive while playing.
  • Audio focus: pause on loss, pause on transient loss and resume on regain, duck on "can duck". ACTION_AUDIO_BECOMING_NOISY (headphones unplugged) pauses.
3. Queue logic shared with web
  • Move the platform-independent parts of web/src/music/player.ts into packages/shared and use them from both apps:
    • gainOf (ReplayGain off/track/album/auto, album gain when an album plays in order, clipping guard)
    • fadeAfter (never crossfade consecutive same-album tracks)
    • item/upcoming/lineUp ordering (repeat, shuffle with shuffledCopy)
    • the server play-queue sync (save, restore, refreshFromServer, SavePlayQueue)
    • loudness measuring ahead (MeasureLoudness)
    • Played / NowPlaying reporting
    • followRemote (used by #48)
  • Storage and engine access are injected (localStorage vs device storage, web Engine vs native module).
  • Web behaviour must not change.
  • Sources: the same rules as player.ts:69 and :194-205. Codecs in DECODES play from /api/music/tracks/{id}/file; everything else, and any failure, falls back to /flac. With #47, a cached file is preferred.
  • The web queue is the server play queue (shared with Subsonic clients), and so is the app's. Opening the app resumes the queue the web left, and the other way round.
4. UI

The web is the spec: music/Library.tsx, routes/album.$id.tsx, artist.$id.tsx, playlist.$id.tsx, music/components.tsx, controls.tsx, Pill.tsx, NowPlaying.tsx, components/MusicSettings.tsx.

  • Music tab:
    • A segmented Albums / Artists / Songs / Playlists view with virtualised grids and lists.
    • Album, artist and playlist pages with hero Tilt (gyro) and the ambient tint (#43 coverTint).
    • Stars, playlists editing, and add-to-queue/play-next through long-press menus.
  • Mini-player (the web's pill): above the tab bar on every tab, tinted from the art.
    • Tap or swipe up expands to Now Playing. Swipe left/right goes to next/previous (dismissThreshold + tick).
    • It can be closed (swipe down) only when paused, like web.
    • It's hidden during video, clips and rooms, and comes back paused.
  • Now Playing (full screen, an expanding shared-element-style transition from the mini-player):
    • artwork with Tilt (touch and gyro) and glare, a scrubber with tick haptics every 10 s, and the transport controls
    • tabs: Lyrics (synced, auto-scrolling, tap a line to seek), Queue (drag to reorder with a tick per slot change, swipe to remove), Info (format, bitrate, sample rate, gain applied, source: file/flac/cache)
    • swipe down to collapse
  • Settings → Music (as web): ReplayGain mode, crossfade, and app passwords for Subsonic clients.
Acceptance criteria
  • A gapless album (e.g. a live album) plays with no gaps. Crossfade works between albums and never inside one. ReplayGain levels match the web for the same track and mode.
  • Music keeps playing with the screen off and the app swiped away from recents (until stopped). The notification, lock screen, Bluetooth buttons and Samsung Now Bar / HyperOS island show and control it.
  • Queue changes on the phone show up in the web UI's queue and the other way round.
  • The web music player still passes the same manual checks (gapless, crossfade, ReplayGain, FLAC fallback, seeking).
  • Opus files play on Android through the opus feature (needed by #47).
Dominant language
TypeScript
Stars
4
Forks
1
Avg merge
4h 49m
Merged PRs (30d)
5

Getting set up

  • Ships a Dockerfile or Docker Compose file
  • No pull request template
  • No contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from tinystream-dev/tinystream

All issues in tinystream-dev/tinystream

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.