The Challenge

I was tasked with producing an intimate, invite-only virtual mass. This meant overlays, showing responses and songs on cue, and controlling which speaker/reader gets shown when.

While there are many commercial solutions out there for streaming and producing such events (such as Restream and StreamYard), in my testing I found them unusable since I had quite the requirements list:

• The biggest crux of all, the event was on Zoom. While solutions for broadcasting a Zoom call to YouTube and other public platforms are a dime a dozen, I have not come across a service that does the reverse: Pipe a produced feed into a Zoom call. This is because the only ways to input to Zoom are either through a virtual camera+mic, or screen share — both of which require a desktop environment.
• It had to be resilient against internet outages. My connection at the time was poor; I could not rely on it to simultaneously download the raw streams from the speakers and upload the produced feed to the Zoom. If I run my setup locally, I run the risk of poor A/V quality and stopping the show entirely when my connection drops.
• As much as possible, I had to avoid having to instruct the speakers how to use new software. Ideally, they'd use Zoom instead of new and unfamiliar software like Skype or VDO.Ninja. Every member of this event, from the speakers to the producers, was working from home — so if something were to go awry, I wouldn't have been able to run to their side and help.
• Whatever stack I ended up using had to be free.

Solution Overview

What I ended up doing was running SLOBS on a maxed-out Windows virtual machine on Google Cloud Compute Engine. This way, I was using the datacenter's network, so in case my local internet went down, at least the show would go on.

I had two Zoom calls: One for the speakers, and another for the actual event. I would screen capture the speakers' Zoom, produce it on SLOBS, then use the virtual camera and virtual audio cable to pipe it into the event Zoom. Controlling which speaker gets seen when was a matter of pinning the appropriate user on the speakers' Zoom.

Remote Desktop Quirks

Specifications

I had to max out my Compute Engine instance (32 cores and 96 GB RAM) simply because I did not want to bother requesting for a GPU; Thus, I had to rely on raw CPU power to render the graphics. This setup worked well for my purposes, with SLOBS' screen capture of the speakers' Zoom working well. What did not work well, however, were browser sources which tools like VDO.Ninja rely on — videos on browsers had dismal frame rates, even with hardware acceleration enabled.

As of the time of writing, Google Cloud offers a $300 90-day credit to new users — which is how I was able to fund an otherwise expensive VM.

Audio Trouble

Setting up audio was a pain. I had difficulty getting my virtual audio cable recognized as an input device on Zoom and other apps. Furthermore, using Microsoft's remote desktop client, audio would get cut if my local machine loses connection with the remote desktop — defeating the purpose of having a cloud VM. What eventually worked was VB-Audio, setting the remote desktop connection to "Play on the remote PC", and finally using TeamViewer. Using TeamViewer was key, as unlike Microsoft's RDC, TeamViewer would keep the audio playing, ensuring that in case I lose connection to the VM, the show goes on.

Video Quirks

One issue I was not able to resolve was the output resolution. If I set it to 720p or above, the output would have parts of it corrupted. I thus had to set it to a measly 540p.

Zoom Quirks

The last hurdle was dealing with Zoom. To recall, I had two Zoom calls: One for the speakers, and another for the actual event. My Zoom license however did not allow joining more than one Zoom meeting at a time on desktop.

To work around this, I used the Zoom desktop app on the VM to host the speakers' Zoom, then joining the event Zoom via a browser, remembering to set the inputs to the SLOBS virtual camera and VB-Audio virtual cable. Note, the event Zoom had to be hosted by someone else. While browser video output was bad, input seemed to work just fine. I monitored the final produced feed by joining the event Zoom from my iPad.

One drawback to using Zoom to manage speakers is that I had to use Zoom's pin feature to control who gets shows when, instead of controlling it directly from SLOBS — which would've been possible had I used another solution like Skype with NDI or VDO.Ninja.

Conclusion

I hope this piece would help others who are tasked to produce Zoom events in poor internet conditions. It would've been easy had I had access to a 100 megabit internet connection — simply running everything locally and avoiding all the pain (and costs) that come with having to run all these in a GPU-less remote desktop that wasn't intended for such use — but that was not the case, and presented quite an interesting challenge to tackle.

While this project was admittedly a massive time drain and incredibly over-engineered considering the final result (540p output and being unable to use SLOBS to control speakers' screen time, when a simple PowerPoint screen share would've sufficed), it was a fun and rewarding project nonetheless. It reminded me of how much joy there is to be found in addressing technical challenges and solidified my passion for making and problem-solving.