Workstation/LiveStreamingCompatInitiative: Difference between revisions

Latest revision as of 14:32, 16 December 2021

Fedora Workstation: Live Streaming Compatibility Initiative

In the last decade the use case for live streaming games, coding sessions, has skyrocketed, specially after the pandemic.

We want to articulate an initiative to make sure that Fedora Workstation has the best out of the box experience for the software and hardware used by this community of users.

In this page we want to summarize the areas we want to cover and articulate a partnership with entities outside of Fedora that have an interest in both upstream and downstream enablement. As an initial effort we probably want to pick single tool in each category and figure out how to make it work well with the rest of the tools. Once we have a golden standard defined then other tools can work to fit into that standard.

A great source of reference for Linux compatible Streaming hardware is the one maintained by Venn Stone over at LinuxGameCast.com

Note: After some feedback it is clear we need to define some use-cases here, while there of course is some overlap the requirements to do news or interview shows are not the same is the requirements if you primarily are a game streamer.

Table with an overview of hardware available/being used
Device Name	Tool Category	Status	Price	Notes
Elgato Key Light Air	Light	Works with fiddling	130 USD	This is not connected to the computer, but it is WIFI enabled and can be controlled over the network.
https://www.behringer.com/product.html?modelCode=P0BJZ Behringer UMXxxxHD series	Mixer/DAC	Works well	Sub 100 USD	The UMC Series of Behringer DAC's are the most affordable DAC's on this list and have better pre-amps than the Focusrite's
https://www.behringer.com/product.html?modelCode=P0ASF Behringer X32	Mixer	Works well	2500 USD	Got official Linux support/app (non-free)
Behringer X12	Mixer	Works well	375 USD	Got official Linux support/app (non-free)
TC Helicon GoXLR	Mixer	Not working	500 USD	Can offer some functionality with severe fiddling - Go-XLR-on Linux
Elgato Stream Deck	Programable Key Pad	Works	120 USD	Not officially supported, but decent open source support
Audio Technica AT2005USB	USB Microphone	Works Well	100 USD	Plug and Play USB Microphone
Blue Yetie	USB Microphone	Works Well	99 USD	Plug and Play Microphone
Logitech Brio 4K	USB Webcam	Works Well	199 USD
Dell UltraSharp Webcam	USB Webcam	Works Somewhat	199 USD	Kinda works, but can easily cause lockups and similar
Magewell USB Capture HDMI 4K Plus	HDMI Capture	Works Well	450 USD
XR1 lite Capture Card USB Capture HDMI 4K Plus	HDMI Capture	Works Well	60 USD

Light

Elgato Key Light Air This is not connected to the computer, but it is WIFI enabled and can be controlled over the network.
Prior art:
- GNOME Shell extension for controlling lamp
- Another shell extension, more basic, but on extensions.gnome.org

Mixer and Input decks

Behringer X32 This one actually got a official Linux GUI application. A bit pricey for the Joe Average streamer perhaps?
TC Helicon GoXLR Not especially well working atm.
Elgato Stream Deck GamingOnLinux article about Stream Deck Using StreamDeck with Fedora
Elgato Wave XLR
Focusrite Scarlett devices
We lack a good-generic key remapping software that could tie any keyboard/gamepad input device into more sophisticade actions. Moving to flatpaks means that we need to think about Portal infrastructure to connect apps with one another. In the case of StreamDeck we may need to create pipewire APIs for an app to do video processing, audio effects, etc... (this needs engagement with the OBS folks)

Some extra input in this twitter thread

High-end USB Microphones

Audio Technica AT2005USB Chris Fisher recommends this one for Linux as a better option than the Blue Yeti.
Blue Yeti Works well with Linux.

High End Webcams

Logitech BRIO Works well under Linux.
Dell SharpEdge Works somewhat with Linux.

DSLR

For DSLR as webcam over USB we rely o non upstreamed v4l2loopback + gphoto2. v4l2loopback is very unlikely to go upstream however libcamera seems to be the way forward. Worth exploring what kind of work is needed to get DSLRs working with libcamera.

Worth exploring if some setups rely on software that configure or control the camera somehow from the OS itself or simply getting the stream is fine for most usecases.

Note: Spoke with Laurent Pinchart of libcamera fame and some sort of libgphoto powered backend for libcamera isn't out of the question. Someone needs to write the code of course and prove that it is viable. If anyone does end up looking at this he did ask they relatively early reach out to discuss architecture and pitfalls.

Smartphone as Webcam

Lots of Smartphones these days have top notch camera sensors, we could take advantage of this:

USB to HDMI adapters
Extend KDE Connect and integrate it with libcamera/PipeWire

Capture

Magewell USB Capture HDMI 4K Plus Works well under Linux.

Software

OBS Studio - We want to reach out to OBS to ensure top notch PipeWire integration and Flatpak availability - Georges Stavracas
Noisetorch Popular tool for cleaning up background noise
Helvum - Patchbay for PipeWire
EasyEffects - Audio effects tool for PipeWire
Steam and Lutris - Gaming often the focus of streamers
Reaper Advanced tool, non-free, but with Linux version
[1] Keymapper for mapping keys on these devices.

Video editing

- DaVinci Resolve - Blackmagic Design's DaVinci Resolve is not open source but still free of charge. It provides video editing, color correction, visual effects, motion graphics and audio post production capabilities on Linux using the same tools as on the other but closed platforms.
- PiTiVi
- Shotcut
- KDEnlive
- OpenShot

Hardware encoding

Worth summarizing what the status is for hw encoding on each of these GPU platforms for both MESA and proprietary drivers and then figuring out whether OBS can leverage those or not.

Intel (VAAPI)
- Example of VAAPI issue
AMD (AMF (currently only available in their proprietary driver stack))
NVIDIA (nvenc)

Partnerships

We intend to gather support from vendors and industry players to build momentum around this initiative.

Device Management

- Control Center Issue #1172: Default audio device and disabling audio devices - When live streaming for various reasons you may need to plug different equipment in/out and different times... this issue makes it more difficult to manage.
- Sound output device chooser GNOME Shell extension is commonly used to quickly and easily switch output devices for getting ready for streaming. Ideally, this should be built into GNOME Shell and manage both input and output devices.

Audio Device behavior

The current behaviour on connecting/disconnecting new audio devices is very frustrating for content creators. This bug should have the details. Note: smarter behaviour here is hopefully something Wireplumber will make easier.

@@ Line 4: / Line 4: @@
 We want to articulate an initiative to make sure that Fedora Workstation has the best out of the box experience for the software and hardware used by this community of users.
-In this page we want to summarize the areas we want to cover and articulate a partnership with entities outside of Fedora that have an interest in both upstream and downstream enablement.
+In this page we want to summarize the areas we want to cover and articulate a partnership with entities outside of Fedora that have an interest in both upstream and downstream enablement. As an initial effort we probably want to pick single tool in each category and figure out how to make it work well with the rest of the tools. Once we have a golden standard defined then other tools can work to fit into that standard.
+A great source of reference for Linux compatible Streaming hardware is the one maintained by Venn Stone over at [https://www.amazon.com/shop/linuxgamecast LinuxGameCast.com]
+Note: After some feedback it is clear we need to define some use-cases here, while there of course is some overlap the requirements to do news or interview shows are not the same is the requirements if you primarily are a game streamer.
+{| class="wikitable"
+|+ Table with an overview of hardware available/being used
+|-
+! Device Name !! Tool Category !! Status !! Price !! Notes
+|-
+| [https://www.elgato.com/en/key-light-air Elgato Key Light Air] || Light || Works with fiddling || 130 USD || This is not connected to the computer, but it is WIFI enabled and can be controlled over the network.
+|-
+| https://www.behringer.com/product.html?modelCode=P0BJZ Behringer UMXxxxHD series || Mixer/DAC || Works well|| Sub 100 USD || The UMC Series of Behringer DAC's are the most affordable DAC's on this list and have better pre-amps than the Focusrite's
+|-
+| https://www.behringer.com/product.html?modelCode=P0ASF Behringer X32 || Mixer || Works well|| 2500 USD || Got official Linux support/app (non-free)
+|-
+| [https://www.behringer.com/product.html?modelCode= Behringer X12] || Mixer || Works well|| 375 USD || Got official Linux support/app (non-free)
+|-
+| [https://www.tc-helicon.com/product.html?modelCode=P0CQK TC Helicon GoXLR] || Mixer || Not working || 500 USD || Can offer some functionality with severe fiddling - [https://github.com/GoXLR-on-Linux/goxlr-on-linux Go-XLR-on Linux]
+|-
+| [https://www.elgato.com/en/stream-deck-xl Elgato Stream Deck]|| Programable Key Pad || Works || 120 USD || Not officially supported, but decent open source support
+|-
+| [https://www.audio-technica.com/en-us/at2005usb Audio Technica AT2005USB] || USB Microphone || Works Well || 100 USD || Plug and Play USB Microphone
+|-
+| [https://www.bluemic.com/en-us/products/yeti/ Blue Yeti]e || USB Microphone || Works Well || 99 USD || Plug and Play Microphone
+|-
+| [https://www.logitech.com/en-us/products/webcams/brio-4k-hdr-webcam.960-001105.html Logitech Brio 4K] || USB Webcam || Works Well || 199 USD ||
+|-
+| [https://www.dell.com/en-us/shop/dell-ultrasharp-webcam/apd/319-bbhp/pc-accessories?gacd=9694607-1002-5761040-0-0&dgc=st&gclid=Cj0KCQiAkZKNBhDiARIsAPsk0Whzt9_s3tByL3eWRIFwWenIvS5PwAAJzWlanRe8YaEBBndaO-iL8kgaAvKjEALw_wcB&gclsrc=aw.ds&nclid=ioRrJGDjfJYHdOL4a8NfK7lyDnh0_8RmGvScCHAc5pZzOAPl8nLtVsbyIxxCALS1 Dell UltraSharp Webcam]|| USB Webcam || Works Somewhat || 199 USD ||  Kinda works, but can easily cause lockups and similar
+|-
+| [https://www.magewell.com/products/usb-capture-hdmi-4k-plus Magewell USB Capture HDMI 4K Plus] || HDMI Capture || Works Well || 450 USD ||
+|-
+| [https://www.evga.com/products/product.aspx?pn=141-U1-CB20-LREVGA XR1 lite Capture Card USB Capture HDMI 4K Plus] || HDMI Capture || Works Well || 60 USD ||
+|}
 === Light ===
-* Elgato Key Light (air)
+* [https://www.elgato.com/en/key-light-air Elgato Key Light Air] This is not connected to the computer, but it is WIFI enabled and can be controlled over the network.
+* Prior art:
+** [https://github.com/Cluster2a/gnome-shell-extension-elgato-light-control GNOME Shell extension for controlling lamp]
+** [https://vidstige.se/gnome-extensions/ Another shell extension, more basic, but on extensions.gnome.org]
 === Mixer and Input decks ===
-* TC Helicon GoXLR
+* [https://www.behringer.com/product.html?modelCode=P0ASF Behringer X32] This one actually got a official Linux GUI application. A bit pricey for the Joe Average streamer perhaps?
-* Elgato Stream Deck
+* [https://www.tc-helicon.com/product.html?modelCode=P0CQK TC Helicon GoXLR] Not especially well working atm.
-* Elgato Wave XLR
+** [https://github.com/GoXLR-on-Linux/goxlr-on-linux Go-XLR-on Linux]
+** [https://github.com/alsa-project/alsa-ucm-conf/issues/121 Upstream ALSA discussion]
+** [https://gitlab.freedesktop.org/pulseaudio/pulseaudio/-/issues/1093 PulseAudio bug thread]
+* [https://www.elgato.com/en/stream-deck-xl Elgato Stream Deck] [https://www.gamingonlinux.com/articles/using-the-elgato-stream-deck-on-linux-just-got-a-whole-lot-easier-with-streamdeck-ui.15180/ GamingOnLinux article about Stream Deck] [https://riferrei.com/using-stream-deck-with-fedora/ Using StreamDeck with Fedora]
+* [https://www.elgato.com/en/wave-xlr Elgato Wave XLR]
+* [https://focusrite.com/en/scarlett?hsa_acc=9998263402&hsa_cam=2047999879&hsa_grp=71929016869&hsa_ad=431498514462&hsa_src=g&hsa_tgt=kwd-44800660&hsa_kw=audio%20interface&hsa_mt=p&hsa_net=adwords&hsa_ver=3 Focusrite Scarlett devices]
+* We lack a good-generic key remapping software that could tie any keyboard/gamepad input device into more sophisticade actions. Moving to flatpaks means that we need to think about Portal infrastructure to connect apps with one another. In the case of StreamDeck we may need to create pipewire APIs for an app to do video processing, audio effects, etc... (this needs engagement with the OBS folks)
 [https://twitter.com/acruiz/status/1463574331301244930 Some extra input in this twitter thread]
+=== High-end USB Microphones ===
+* [https://www.audio-technica.com/en-us/at2005usb Audio Technica AT2005USB] Chris Fisher recommends this one for Linux as a better option than the Blue Yeti.
+* [https://www.bluemic.com/en-us/products/yeti/ Blue Yeti] Works well with Linux.
+=== High End Webcams ===
+* [https://www.logitech.com/en-us/products/webcams/brio-4k-hdr-webcam.960-001105.html?utm_campaign=dr&utm_source=google&utm_medium=pla&cvosrc=cse.google.%7Bkeyword%7D&cvo_campaign=brand&cvo_crid=%7Bcreative%7D&cvokid=%7Bprofile_id%7D%7C%7Baff_code%7D&gclid=%7Bgclid_value%7D&prodsku=%7Bproduct_id%7D#buy Logitech BRIO] Works well under Linux.
+* [https://www.dell.com/en-us/shop/dell-ultrasharp-webcam/apd/319-bbhp/pc-accessories?gacd=9694607-1002-5761040-0-0&dgc=st&gclid=Cj0KCQiAkZKNBhDiARIsAPsk0Whzt9_s3tByL3eWRIFwWenIvS5PwAAJzWlanRe8YaEBBndaO-iL8kgaAvKjEALw_wcB&gclsrc=aw.ds&nclid=ioRrJGDjfJYHdOL4a8NfK7lyDnh0_8RmGvScCHAc5pZzOAPl8nLtVsbyIxxCALS1 Dell SharpEdge] Works somewhat with Linux.
 === DSLR ===
@@ Line 20: / Line 71: @@
 Worth exploring if some setups rely on software that configure or control the camera somehow from the OS itself or simply getting the stream is fine for most usecases.
+Note: Spoke with Laurent Pinchart of libcamera fame and some sort of libgphoto powered backend for libcamera isn't out of the question. Someone needs to write the code of course and prove that it is viable. If anyone does end up looking at this he did ask they relatively early reach out to discuss architecture and pitfalls.
+=== Smartphone as Webcam ===
+Lots of Smartphones these days have top notch camera sensors, we could take advantage of this:
+* USB to HDMI adapters
+* Extend KDE Connect and integrate it with libcamera/PipeWire
 ===Capture===
-* Magewell USB Capture HDMI 4K Plus Device
+* [https://www.magewell.com/products/usb-capture-hdmi-4k-plus Magewell USB Capture HDMI 4K Plus] Works well under Linux.
 === Software ===
-==== OBS ====
+* [https://obsproject.com/ OBS Studio] - We want to reach out to OBS to ensure top notch PipeWire integration and Flatpak availability - Georges Stavracas
-We want to reach out to OBS to ensure top notch PipeWire integration and Flatpak availability.
+* [https://github.com/lawl/NoiseTorch Noisetorch] Popular tool for cleaning up background noise
+* [https://gitlab.freedesktop.org/ryuukyu/helvum Helvum] - Patchbay for PipeWire
+* [https://github.com/wwmm/easyeffects EasyEffects] - Audio effects tool for PipeWire
+* [https://store.steampowered.com/ Steam] and [https://lutris.net Lutris] - Gaming often the focus of streamers
+* [https://www.reaper.fm/ Reaper] Advanced tool, non-free, but with Linux version
+* [https://github.com/sezanzeb/key-mapper/] Keymapper for mapping keys on these devices.
+Video editing
+** [https://www.blackmagicdesign.com/products/davinciresolve/ DaVinci Resolve] - Blackmagic Design's DaVinci Resolve is not open source but still free of charge. It provides video editing, color correction, visual effects, motion graphics and audio post production capabilities on Linux using the same tools as on the other but closed platforms.
+** [https://www.pitivi.org/ PiTiVi]
+** [https://shotcut.org/ Shotcut]
+** [https://kdenlive.org/en/ KDEnlive]
+** [https://www.openshot.org/ OpenShot]
 === Hardware encoding ===
-Worth summarizing what the status is for hw encoding on each of these GPU platforms for both MESA and propietary drivers and then figuring out whether OBS can leverage those or not.
+Worth summarizing what the status is for hw encoding on each of these GPU platforms for both MESA and proprietary drivers and then figuring out whether OBS can leverage those or not.
-* Intel
+* Intel ([https://github.com/intel/libva VAAPI])
-* AMD
+** [https://gitlab.freedesktop.org/mesa/mesa/-/issues/3540 Example of VAAPI issue]
-* NVIDIA
+* AMD ([https://gpuopen.com/advanced-media-framework/ AMF] (currently only available in their proprietary driver stack))
+* NVIDIA ([https://obsproject.com/forum/threads/nvenc-support-for-linux.47023/ nvenc])
 === Partnerships ===
 We intend to gather support from vendors and industry players to build momentum around this initiative.
+=== Device Management ===
+** [https://gitlab.gnome.org/GNOME/gnome-control-center/-/issues/1172 Control Center Issue #1172: Default audio device and disabling audio devices] - When live streaming for various reasons you may need to plug different equipment in/out and different times... this issue makes it more difficult to manage.
+** [https://extensions.gnome.org/extension/906/sound-output-device-chooser/ Sound output device chooser GNOME Shell extension] is commonly used to quickly and easily switch output devices for getting ready for streaming. Ideally, this should be built into GNOME Shell and manage both input and output devices.
+=== Audio Device behavior ===
+* The current behaviour on connecting/disconnecting new audio devices is very frustrating for content creators. [https://gitlab.gnome.org/GNOME/gnome-control-center/-/issues/1172 This bug should have the details.] Note: smarter behaviour here is hopefully something Wireplumber will make easier.

Search