I like WebRTC but it's a complicated mess of protocols...
I'm currently trying to implement a WebRTC peer client in pure Erlang. One of the most difficult things I've done because WebRTC relies on several hundred pages of RFCs.
Yup, lots of telephony-inspired RFCs in there. If you're not already, I suggest referencing [0] as you develop, it's a great project that is also really easy to read what's happening underneath.
Thank goodness for pions webrtc! I changed my MMO's dev version over to it a couple months ago. This is going to be tons better than the node-electron webrtc proxies running under tmux I was using. Between this, nanomsg, and BadgerDB, I'm going to be removing all of my non-golang dependencies, which is going to enable some very cool stuff.
For one thing, I now have edge servers in my architecture. This means that I can now use my ability to serialize the state of a game instance server process, then inject it into another process without any participation from the client. This will make hot code updates really slick.
We have had a lot of churn, but I am really proud of how we split everything out. DTLS, SCTP, ICE and SRTP are a real pain but once it works it is amazing.
We are also working on https://pion.ly/knowledge-base I want to create a resource that pulls all WebRTC info together RFCS, summations and links to relevant code etc...
If anyone wants to help we would love to have you! Especially someone who has an eye for design, we have a resource but they aren't able to help often, I am always happy to chat in our Slack channel https://pion.ly/slack/
----
I still really want to tie your IFPS work together we something like https://github.com/pions/asciirtc then we can get decentralized chat that works in browsers and desktops (with a single static binary to run it locally) would be amazing
If there is anything I can do I would like to support your Erlang implementation! I can answer any questions you have, and would be happy to help get more eyes on your version as well.
It would be fantastic to see Go, Python, Erlang, C++ and Browsers all communicating directly without any software in the middle :)
How do you handle DTLS in Erlang?
The Erlang DTLS implementation in OTP is missing the use_srtp extension and muxing DTLS, RTP and STUN packets on the same port.
I'm still not sure how to do the DTLS handshake. I was under they impression that I might be able to use the ssl:handshake and then downgrade again, but I realize reading your comment it will be much more difficult...
I will do that. Checked the archives after your comment and it seems use_srtp support is planned. My current understanding is that for dtls-srtp only the handshake is necessary for key exchange.
I wish you could follow duplicates of radars like in github.. That issue was "resolved" by saying its a duplicate of issue 29281220, which isn't helpful at all. That issue could be even older than the one you linked.
If this is something you care about, you should file an enhancement request, even if you write that it is a duplicate of that issue yourself.
Right now, WKWebView and SFSafariViewController (and the authentication service variant, and Web.app which is used for shortcuts on the home screen) do not have WebRTC support - only MobileSafari.app does.
The model I was thinking of was native code in the native app, injected polyfill that talks to the native code over a message bus. The native code can draw over the top of the web view and pass messages in. Is there anything in particular stopping that?
I've heard mixed reports on this - the big issue is technical in that you can't implement your own JIT Javascript engine, because only JavaScriptCore (running out-of-process) can write executable pages.
> The VP8 video codec is widely used in existing WebRTC solutions. It is now supported as a WebRTC-only video codec in Safari 12.1 on both iOS and macOS betas.
For Steve's sake! Why not add VP8 and OPUS system-wide already when they already even are built in different parts of the system?
probably because Apple doesn't have hardware acceleration for vp8. They don't want developers to use it. It'll be an inferior UX. Less battery, hotter device.
It's a bit arrogant for them to compare battery life of h264 to vp8, as they've chosen not to support hardware accelerated vp8. All Android devices have had this for years.
I'm hoping this all settles out with av1. Netflix and Amazon will support av1, so that doesn't leave Apple with much choice.
The Intel chips that Apple uses support VP8 decoding since Broadwell and encoding since Cherryview/Braswell. On the mobile side, it was their conscious decision not to support it, just like they newer supported the free audio codecs or container formats either.
Because Apple had significant stakes in MPEG-LA and VP8 and its descendants were a significant threat to its licensing revenues. Thankfully the HEVC fiasco and AV1 have forced them to accept that MPEG is heading towards irrelevance, but I don't expect them to implement support for VPx codecs any time soon. The only thing left is to wait for AV1 to mature and begin being implemented in hardware.
Honestly I don't care much about AV1, I am hardly going to own AV1-capable hardware in less than 5-10 years anyway. What I really want is OPUS to work everywhere (HTML audio tag in web browsers, message-attached audio files in messengers (Telegram), iTunes) and possibly VP8(webm)-encoded "gifs". I have converted my whole lifetime collection of MP3s (some hundreds gibs) to OPUS to save space (and put a big collection of audio books on my Android phone) and now I have to convert them back every time I want to let my wife listen to something on her iPhone, iPad or MacBook (there is VLC for this on Mac yet lack of web browser support is still a pain).
I've seen many mentions OPUS is going to be everywhere, Apple included, years passed and it still is not, not even in the fresh models that were meant to have it in hardware.
Opus works in macOS natively in QuickTime, but only if contained in a CoreAudioFormat container (.CAF). Tested on macOS Mojave, it works. They simply never implemented any kind of support for OGG.
macOS us the least of a problem. I don't even see a reason to use QuickTime when there is VLC that supports everything. The real deal are iOS and web browsers on both macOS and iOS. IIRC I've tried CAF in Chrome and Safari but neither worked.
Sadly until they implement the OGG container there isn't a lot to do about Opus on iOS. They could also gasp consider adding support for Vorbis and Theora, but I think they would rather stop making computers than do it at this point.
All I know about WebRTC is Chrome preventing sleep on my systems with "WebRTC has active peer connections". Made me ditch Chrome before it was fashionable to de-google.
So the only new feature i want from this WebRTC thingy is... an OFF button.
Fortunately, Safari's WebRTC won't silently leak your private IP address or other network info. We've also proposed a new protocol extension that enables peer-to-peer pure data connections in a privacy-aware way.
As a user, I really appreciate how the WebKit team approaches new features in such a measured and careful way. New shiny things to use are great but I’m more than willing to wait if it helps to prevent glaring oversights. There’s no benefit to rushing these things in.
isn’t webrtc by default off on all browsers? aren’t websites asking for permssion before the js is allowed access? and won’t it stay off when you deny the request?
ah, i believe you are referring to the webrtc data channel. it leaks local IPs, but the severity depends on several factors, including whether you're running VPN and what you're using the VPN for, or just running behind a regular local network.
if you're running behind a regular local network then I wouldn't consider the local IP leakage as a "privacy hazard". local IPs are compromised already. everywhere. they are easy to guess. they are easy to obtain in native apps. etc.
there are issues when it comes to places where VPN access is crucial/vital. thankfully, very few VPN providers leak your IP nowadays, and with drafts such as what the poster above mentioned (https://datatracker.ietf.org/doc/draft-ietf-rtcweb-mdns-ice-...) this problem will be history soon enough.
VP8 can only be contained in MKVs and WebMs files, so I do see this happening. Apple is also part of the consortium working on AV1, the successor to VP8 and VP9; it also uses the MKV and WEBM containers.
The AOM membership and the VP8 support may even be related. If Apple's assuming risk from submarine patents, etc. inherent in shipping an AV1 implementation, VP8 is something like a subset of it, so they can get that "for free".
The post talks about how iPhone 7's have worse battery life with VP8, which I'm sure is true, but it's still kinda funny because...it's Apple quasi-complaining here, and Apple who can solve the lack of acceleration, at least for future products.
AVC1 is just H264[0]. FFmpeg says stream 0 is "Video: h264"
The page you linked about AV1 in mp4 says it uses MPEG-4 Part 31, which is under development and not final [1]. Until now I was not aware they were extending the MPEG-4 container standard to include more formats.
Apple has released software decoders before, but you might be right about waiting for hardware decoders first. They released an HEVC software decoder for devices that could get iOS 11, but had a chip older than the A9, (like the iPhone 5S and 6).
> Safari also comes with additional improvements, including better support of capture device selection, experimental support of the screen capture API, and deprecation of the WebRTC legacy API.
Looking very much forward to this. Currently my plugin-free in-browser screen-sharing tool [0] only works in FF and Chrome. The real question is will mobile browsers offer it. In general though, I am impressed with how well screen capturing over WebRTC works and so far haven't seen that many people needing a TURN server.
As someone who harassed you about VP8 support in the past [0], this is really great news. Thank you for all of your hard work. I am also really happy to see the transition to Unified Plan.
Free software. The Cisco H.264 codec supplied with Firefox cannot be distributed with other software, despite being "open". From https://www.openh264.org/ , there seem to be licensing costs involved with the usage of this code, which have been waived for some uses, implying they were not waived for others:
> We will not pass on our MPEG-LA licensing costs for this module, and based on the current licensing environment, this will effectively make H.264 free for use on supported platforms.
EDIT: To clarify, such licensing requirements on H.264 may make codecs fall outside of the guidelines of software distributions allowing only free software, making them supply their browser without one.
If I remember correctly, only decoding for online video streaming doesn't require licensing fees. Cisco hits the fee cap with the products they sell already, so letting everyone use their binaries doesn't cost them anything.
Also includes lots of people w/ custom Chromium-based browsers that don't ship with these codecs (and libs like CEF don't compile with them enabled by default).
Many RTC stacks (and specifically openh264) also only support H.264 baseline profile. So you are likely to get higher quality with VP8 (subject to variations in the quality of the implementations on each endpoint, of course).
Some services use WebRTC in a "hub and spoke" model rather than true peer-to-peer, and they often require VP8 due to a desire to support low-end Android devices that don't have hardware H.264 decoding.
I am surprised to hear that. Even $50 Android Phones now has H.264 hardware decoding support, what phones are those that doesn't come with it? And likely doesn't have VP8 hardware decoding either.
This is super exciting, so many cool things happening here.
I haven't had a chance to grok it fully yet but someone added WASM support to pion recently [0] you will able to just write your code once in Go and should work in both places!
I'm currently trying to implement a WebRTC peer client in pure Erlang. One of the most difficult things I've done because WebRTC relies on several hundred pages of RFCs.