Webkit browsers can't share video (Safari, all iOS browsers) #14
Labels
No Label
bug
duplicate
enhancement
help wanted
invalid
question
wontfix
No Milestone
No project
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: apps/BareRTC#14
Loading…
Reference in New Issue
There is no content yet.
Delete Branch "%!s(<nil>)"
Deleting a branch is permanent. Although the deleted branch may exist for a short time before cleaning up, in most cases it CANNOT be undone. Continue?
Browsers using the Webkit engine all seem to have difficulty connecting to peers to enable webcam sharing. Some browsers that currently have issues:
What's known about the issue
A user on Safari is able to turn on their own camera (and so their cam icon lights up blue for others to click), but when others click on their camera, nothing happens; similarly, when the Safari user clicks on somebody else's camera, nothing happens.
From my Firefox browser on Linux when I try and connect to the Safari user, my console says that peer-to-peer WebRTC connection fails and there is no TURN server to carry the video instead (the latter of which is expected - the chat room does not have a TURN server). update: I added TURN server support but it has not improved the situation.
I have not seen what the error is on Safari's side. At one point I saw a console error that Safari was unhappy that the ordering of lines for the WebRTC handshake was being altered (because the server parsed and re-serialized the json messages), but fixing that (passing the messages verbatim) did not resolve the issue, and Safari on a macOS Ventura virtual machine logged no error at all in the console for me.
Linux Webkit Browsers
In both GNOME Web (Epiphany) and Midori (Xfce's minimalist browser), opening a user's webcam fails because seemingly these Webkit browsers do not support WebRTC at all? They give a console error: "Variable not found: RTCPeerConnection"
Things to do
Additional Context
How webcam sharing generally works in BareRTC (see also: chat server protocol):
navigator.mediaDevices.getUserMedia()to prompt the user to share their webcam and microphone.<video>element directly sourced by their webcam stream, with its audio muted so you don't hear your own microphone echo from it.At one point in step 3 I saw Safari desktop's error console complain that the ICE messages (json blobs) were rearranged from how it sent them (because the chat server would parse the json message, and re-serialize it when sending it to the peer - the data was the same but keys got reordered, but I updated it so the chat server just forwards the json blob verbatim without messing with it - this resolved that Safari error, but then Safari still doesn't connect and I couldn't get another error message from it yet).
With PR #36 we got iPads able to connect!
But if the non-iPad initiates the call, it doesn't go thru and they get this error from ChatClient:
This looks similar to an error seen long ago by Safari, and seemed caused by the chat server parsing and re-serializing the SDP and Candidate messages and Safari being unhappy about it. The server switched to just passing the messages along verbatim, and Safari stopped throwing this error.
"Apple compatibility mode"
In
8e87c377e8a breakthrough was found that enabled Webkit browsers to open other peoples' cameras, "most of the time."The chat added an "Apple compatibility mode" checkbox that enabled this new behavior - on by default if the chat page could detect you were on an iPad or iPhone, and enable-able manually if the auto-detection failed.
When enabled, how it would work is this:
The Apple compatibility mode controls whether in step 2 the local video is always sent on the request - it is more efficient not to send your video if the receiver is going to ignore it, and is only a necessary workaround for Webkit viewers.
But: there is still a bug (likely in my own code) where connecting in to view a Webkit browser's camera doesn't work; the Webkit browser has to initiate the call for it to go through. And Webkit-to-Webkit connections probably don't work either.
New workaround ideas for better functionality
Scenario 1
Say you are on a Firefox-on-Windows browser and:
Normally, you opening their cam won't work but since your page was technically already receiving their video (but ignoring it), the page could check if a video stream is already coming in and just show it on the page (and it doesn't even need to begin WebRTC negotiations, it can skip straight to showing the video feed it previously was ignoring).
Scenario 2
Say we have two users on camera in the chat:
And both users are on camera. Alice wants to open Bob's camera, but this won't normally work because Webkit needs to be the one to start the connection. How to work around it?
By sending a message via the chat server, Alice could tell Bob's page to start the connection from his side instead, so that the video connect might actually go through, e.g.:
This would basically emulate the regular workflow of: Alice needing to enable "auto-open my viewer's cam" and then asking Bob to open her camera. But Alice wouldn't need to auto-open videos, and from her perspective she was able to "just open" Bob's camera like normal as if he wasn't on a Webkit browser at all.
* The chat server would need to know who has Apple compatibility mode enabled, which is currently a front-end-only concern. So when a non-Webkit browser wants to connect in to a Webkit browser, the chat server can send specialized messages to both sides in order to reverse the connection workflow to make it happen.
This would probably still not solve Webkit-to-Webkit connections, but should enable non-Webkit users (who are on camera themselves - this is always a requirement) to open a Webkit user's camera from their side (with possible auto-open on the Webkit end working too).