Server-side Speaking Indicators #56

New issue

Open

opened 2026-04-30 22:30:32 +00:00 by kirsle · 0 comments

kirsle commented

2026-04-30 22:30:32 +00:00

Owner

Related to/a possible solution for #55.

There could be a server-side signal (broadcast to all chatters) to show when certain webcam broadcasters are actually speaking/sending audio over their microphone.

Currently:

Webcams that you are currently watching will show a visual indicator when audio is coming in from the camera, even if the camera is muted locally by the viewer.
- The video feed on screen gets a blue border, and a blue speaker icon appears by their name.
This audio cue is processed locally in your browser by inspecting the audio stream coming in from WebRTC by using hark.js
Therefore, you can only tell when the people who you are already watching are speaking, but not when other webcams have audio being sent.

When you are watching somebody who is having a voice chat with others (and you don't have the others' cameras open), having a global server-side speaking indicator can help narrow down which other camera(s) they may be conversing with.

Implementation

To minimize network chatter, the feature should make use of debounces and regular 'heartbeat' style updates.

From the webcam broadcaster's side, have the client send a mic message over WebSocket:
```
{"action": "mic"}
```
- The message would be sent when Hark.js has detected audio coming from the current user's own microphone.
- The message would be debounced (e.g., broadcaster sends the message not more than once per 5 seconds. If they are monologuing, the message would be sent on a 5-second interval and would stop being sent when their mic goes quiet for a period of time.
- The message should be short (no parameters) to reduce network chatter.
From the server back-end:
- Maintain a map of usernames to their latest "speaking" status.
- Each "speaking" status would have an expiration time (5 seconds out) -- if a new "mic" event is not received then their entry in the map will be removed.
The server broadcasts the speaking states periodically.
- As long as any username currently has their mic flag set,
- The server would broadcast a dedicated (short) message communicating that state to end users.
- The message might go out on a 5-second or so interval.
```
{"action": "mic", "usernames": [ "alice", "bob" ]}
```
The chat front-end page would use these messages to indicate which webcams are speaking.
- The page would have its own debounce/expiry time (e.g., when it hears "alice" was speaking, to keep that state for 5 seconds and clear it unless the server has refreshed the state).
- The user entries on the Who's Online list would show an indicator of speaking.

Ideas for how the visual indicator could look:

Stylize the Video button itself?
- Current states and styles for the video button can include:
  - Blue outline and blue video icon (for non-explicit webcams).
  - Blue outline and "eye" icon (camera you are watching).
  - Red outline and red video icon (for explicit webcams).
  - Red outline and "eye" icon
  - A green outline (for invited cams) with a red or blue icon (video or eye icon).
- For "speaking" maybe the background color of the icon can change, or a glow effect added?
Addition of a new icon by their username?
- Currently: a username can have an Operator icon, VIP icon, or custom flair icon.
- Maybe: a small speaker icon (the one shown on videos themselves when audio is detected) is additionally added to their username.

Related to/a possible solution for #55. There could be a server-side signal (broadcast to all chatters) to show when certain webcam broadcasters are actually speaking/sending audio over their microphone. Currently: * Webcams that you are currently watching will show a visual indicator when audio is coming in from the camera, even if the camera is muted locally by the viewer. * The video feed on screen gets a blue border, and a blue speaker icon appears by their name. * This audio cue is processed locally in your browser by inspecting the audio stream coming in from WebRTC by using hark.js * Therefore, you can only tell when the people who you are already watching are speaking, but not when other webcams have audio being sent. When you are watching somebody who is having a voice chat with others (and you don't have the others' cameras open), having a global server-side speaking indicator can help narrow down which other camera(s) they may be conversing with. ### Implementation To minimize network chatter, the feature should make use of debounces and regular 'heartbeat' style updates. 1. From the webcam broadcaster's side, have the client send a `mic` message over WebSocket: ```json {"action": "mic"} ``` * The message would be sent when Hark.js has detected audio coming from the current user's own microphone. * The message would be debounced (e.g., broadcaster sends the message not more than **once per 5 seconds**. If they are monologuing, the message would be sent on a 5-second interval and would stop being sent when their mic goes quiet for a period of time. * The message should be short (no parameters) to reduce network chatter. 2. From the server back-end: * Maintain a map of usernames to their latest "speaking" status. * Each "speaking" status would have an expiration time (5 seconds out) -- if a new "mic" event is not received then their entry in the map will be removed. 3. The server broadcasts the speaking states periodically. * As long as any username currently has their mic flag set, * The server would broadcast a dedicated (short) message communicating that state to end users. * The message might go out on a 5-second or so interval. ```json {"action": "mic", "usernames": [ "alice", "bob" ]} ``` 4. The chat front-end page would use these messages to indicate which webcams are speaking. * The page would have its own debounce/expiry time (e.g., when it hears "alice" was speaking, to keep that state for 5 seconds and clear it unless the server has refreshed the state). * The user entries on the Who's Online list would show an indicator of speaking. Ideas for how the visual indicator could look: * Stylize the Video button itself? * Current states and styles for the video button can include: * Blue outline and blue video icon (for non-explicit webcams). * Blue outline and "eye" icon (camera you are watching). * Red outline and red video icon (for explicit webcams). * Red outline and "eye" icon * A green outline (for invited cams) with a red or blue icon (video or eye icon). * For "speaking" maybe the background color of the icon can change, or a glow effect added? * Addition of a new icon by their username? * Currently: a username can have an Operator icon, VIP icon, or custom flair icon. * Maybe: a small speaker icon (the one shown on videos themselves when audio is detected) is additionally added to their username.

kirsle added the

enhancement

label

2026-04-30 22:30:32 +00:00