iRacing-STT-VR

Update – Version 1.9 is here! See the bottom of this post for the changelist.

Hello, fellow racers!

I am hard of hearing, and in a former life I was also a game developer (Bethesda Softworks, Rainbow Studios, etc.) I have about 50% hearing loss in both ears, and while I can crank my speakers up to hear everything, actually understanding what people are saying via radio chatter is another matter. So… I have created a speech-to-text (STT) overlay app for iRacing. It works beautifully – whenever someone is talking, the app does speech-to-text processing and displays what they are saying in text. It is using Microsoft Azure’s Cognitive Services to do the speech-to-text processing.

This app displays some basic telemetry (your car number, current position, laps behind the leader, current speed, and current gear) at the top, and the speech to text stuff is at the bottom. You can place this overlay anywhere – I usually place it as high as possible on top of the car dash where it doesn’t block the view of anything, so I don’t need to move my eyes too far to read stuff.

Originally this app was developed supporting only OpenVR. But now, as of version 1.6, it also supports monitors as well! You must play iRacing in one of the windowed modes for this to work (borderless window is fine).

Here is a short video showing it off –

I am making this freely available to anyone who wants to use it. It does require some configuration, and I am going to try my best to write clear instructions on how to set it up and use it.

Also, the source code is public and available on GitHub:
https://github.com/mherbold/iRacing-STT-VR

Installing

Make sure you uninstall older versions of this app before installing the newer version – the installer isn’t properly configured to support in-place updates.

First, grab it here – https://herboldracing.com/files/iRacing-STT-VR-1.9.zip

Inside that ZIP file is the setup program for this app, called iRacing-STT-VR-Setup.exe. Just run that setup program to install this app on your computer.

After installing the app, you can run it by launching it from the Windows start menu. If you get to this point, you should see the iRacing-STT-VR app window.

iRacing-STT-VR Settings

Status

This is the status tab, which should you should look at for any error messages.

Cognitive Service API

This is the cognitive service API tab, and this is where you should put in your Azure Cognitive Service Key and Cognitive Service Location / Region. See the Microsoft Azure section below for more information.

Speech Service

This is the speech service tab, where you can configure options for the speech service. There are currently two options –

  • Log File Name – If this is blank, then the speech service log file is disabled. If this is not blank, then the speech service will write out some pretty detailed information to this file. This is useful for troubleshooting issues with the speech service.
  • Enable Profanity Filter – Self explanatory.
Graphics

This is the graphics tab, where you can configure options for the visual elements of this app.

  • Mode – by default this is set to desktop window overlay mode. You can change this to OpenVR overlay mode.
  • Background Texture File Name – by default this is background.png, which is the background texture that I made. You can replace this with your own custom background texture. Be aware that the width and height of the overlay that shows up in the VR headset will match exactly the width and height of this texture. I find that a 1024×256 texture works well.
  • Background Opacity – This controls the transparency of the background image. If you slide this all the way to the left, the background image will be completely invisible.
  • Font File Name – give this the name of the font file that you want the app to use to draw text on the overlay. The default font file name is RevolutionGothic_ExtraBold.otf.
  • Font Size – use this slider to increase or decrease the size of the text that is drawn on the overlay.
Controllers

This is the controllers tab, where you can select any joystick device that you have plugged into your computer to control the position and size of the overlay. If this is set to None, you will not be able to control the position and size of the overlay.

Audio

This is the audio tab, where you can select a couple of options –

  • Audio Capture Device Name – this is the device you want the speech-to-text service to listen on. This should be the CABLE Output device! See the VB-CABLE section below for more information.
  • Audio Render Device Name – this is the device you want to mute and unmute automatically, when you press the push-to-talk button in iRacing. If you race with very loud speakers (like I do!) then you want to use this option to mute those speakers when you talk, so that other people aren’t being blasted with your game audio while you talk.

Microsoft Azure

You must create a Microsoft Azure account and enable the speech service on the account. You get 5 hours of free speech-to-text transcription a month. This does not mean you are limited to 5 hours of iRacing before you have to pay Microsoft for speech-to-text. My app only transmits data to Azure when someone is actually talking. So if no one talks for the whole race, it’s zero minutes used up.

  1. Create a Microsoft Azure subscription:
    https://azure.microsoft.com/free/cognitive-services
  2. Create a speech resource in the Azure portal:
    https://portal.azure.com/#create/Microsoft.CognitiveServicesSpeechServices
  3. Click on “Next steps > Go to resource” to get the keys for your speech resource. You will need two things – the “Key” and the “Location/Region”.

For more help with getting the key and region for the cognitive service API settings – please check out this YouTube video –

https://www.youtube.com/watch?v=BYd3C16SQaA

And skip to around 2:35 where she shows you how to add speech services to your Azure account.

On the “Create Speech Services” screen just create a new resource group – name it whatever (I just called mine “my_resource_group”), and under instance details, give it a name like “iRacing-STT-VR”. The names don’t matter, it’s just for you to recognize what it is for. For the region just pick a spot close to where you are. For the pricing tier if you pick “free”, you’re limited to a certain number of minutes per month, but you can start with that just to get everything working and switch to the paid tier later.

The important part is at 4:06 where she clicks on “Keys and Endpoint” – On that screen there is a “Show Keys” button – click on that to get your keys and you’ll also need the location/region. It doesn’t matter whether you use Key 1 or Key 2, just grab one of those and put that, and the region, in the cognitive service API settings tab in the iRacing-STT-VR app.

VB-CABLE

You will need to get and install the VB-CABLE Virtual Audio Device.

  1. Download it from here (the big orange Windows icon):
    https://vb-audio.com/Cable/index.htm
  2. Install the audio drivers by running the setup program, and rebooting your computer.

You will need to configure the new CABLE Output device. You can skip the rest of this step now, if you do not want to hear the radio chatter at all. The below instructions are for Windows 11.

  1. Open up the Window system settings (Press the Windows key + I)
  2. Click on “System”
  3. Click on “Sound”
  4. Click on “More sound settings”
  5. Click on the “Recording” tab
  6. Find the “CABLE Output” device and select it
  7. Click on the “Properties” button
  8. Click on the “Listen” tab
  9. Enable the “Listen to this device” option and select the sound device you want to hear the radio chatter on.

iRacing Settings

At this point, you are ready to go. Just run the iRacing and my app at the same time. You should at least see an overlay somewhere in the world when you put on your VR headset. It will say “iRacing is not running.” on the overlay if you are not actually in iRacing yet.

When iRacing is running, hop into the options screen and open up the sound options. Under Voice Chat, find the Playback: option and make sure to pick the CABLE Input device. This is key to getting speech-to-text to work.

This is what actually happens –

  1. Someone presses their push-to-talk button and starts talking.
  2. iRacing sends their voice to the “CABLE Input” sound device.
  3. The “CABLE Input” device mirrors the audio into the “CABLE Output” microphone device.
  4. You will hear the voice if you had done the optional steps to set up the “Listen to this device” property on the “CABLE Output” device.
  5. My app sends the audio from the “CABLE Output” device to Microsoft Azure for speech-to-text processing.
  6. Any transcription I get back from Microsoft is displayed on the overlay in real time as it is processed.
  7. The person releases their push-to-talk button, and my app stops sending audio to Microsoft Azure.

Positioning

To control the positioning of the overlay using your selected joystick device, you would hold down either the first, second, or third buttons on the joystick and move the stick.

  • First button – moves the overlay on the X (left/right) and Z (in/out) axis.
  • Second button – moves the overlay on the Y (up/down) axis.
  • Third button – scales the overlay up and down.

The app will remember your favorite overlay position separately for each car that you race in, into the “Settings.xml” file.

Telemetry

The overlay has some telemetry data at the top, which I have found very useful for my racing experience.

From left to right, they are –

  1. Your car number
  2. Your current lap, the total laps, and estimated laps remaining for fuel
  3. Your car position and the total number of cars for the race
  4. How many laps you are behind the leader. If this says 0.000 you are the leader. If this says 1.500 this means you are one and a half laps behind the leader.
  5. Your speed
  6. Your current gear (R, N, 1, 2, 3, etc…)

Troubleshooting

If you get stuck, the first thing you should do is look at is the iRacing-STT-VR.log file. To find this file, open up the Windows explorer (hit Win+E) and type in %appdata%\iRacing-STT-VR in the bar at the top.

A normal log file looks like this –

Loading settings… OK!
Creating window… OK!
Creating graphics device… OK!
Loading background texture… OK!
Creating render target texture and frame buffer… OK!
Creating Dear ImGui renderer… OK!
Adding font… OK!
Binding background texture to Dear ImGui… OK!
Initializing OpenVR… OK!
Scanning for joystick devices… OK!
Initializing the selected joystick device… OK!
Scanning for audio capture devices… OK!
Initializing the selected audio capture device and speech services… OK!
Scanning for audio render devices.. OK!
Initializing the selected audio render device… OK!

If you ever see any other messages, it is likely indicating some sort of error. Please copy and paste your log into the iRacing Forums thread (see feedback section below), and I will look at it and try to help you out.

Feedback

I welcome any feedback, however please keep in mind that this is a hobby for me. I will definitely add features to this app that makes my racing experience better. I might add features that you want, which doesn’t benefit me, if I am motivated to do so. You can also contribute to this project, if you are a programmer. Any feedback should go to the iRacing Forums here:

https://forums.iracing.com/discussion/comment/330119

Donations

Donations are not necessary, but appreciated! If you do want to donate, you may do so in the form of iRacing credits. Choose “Gift iRacing Dollars” from the “Purchase iRacing Dollars” screen members.iracing.com. Here is a link to that page:

https://members.iracing.com/membersite/member/Recharge.do

Changelog

Version 1.9 – 5/27/2023

  • Fixed rendering of text and images – edges should be a bit less jaggy.
  • Auto-hide the speech-to-text messages after approximately 10 seconds for a more distraction-free racing experience.
  • Hide the overlay when you are not on the track in live sessions.
  • New shift indicator light with RPM.
  • Fixed some minor bugs.

Version 1.8 – 5/15/2023

  • This app now creates a chat log file that saves all of the chat data. This will be used later in a future iRacing-TV app update (check that app out here – https://herboldracing.com/blog/iracing/iracing-tv/)
  • Added a background opacity slider – you can now control the opacity of the background image.

Version 1.7 – 4/28/2023

  • Improved default background texture.
  • Added session flags (caution, green, black, white, checkered, etc.) at the top of the overlay.
  • Fixed issue with re-initializing the window when “Apply Changes” was pressed.
  • Some other minor improvements.

Version 1.6 – 4/26/2023

  • New – Monitor support! Non-VR users can rejoice and sing songs about my uber-awesomeness, and name their children after me now. You’re welcome.

Version 1.5 – 4/26/2023

  • New – display estimated number of laps remaining according to your current fuel level vs. the highest fuel consumption rate over the previous 5 laps.
  • New – car high and low indicators. A dark green bar will be drawn on the left and right sides of the overlay when there are no cars there. A single bright red bar will be drawn if there is a car there. A single bright red bar plus a flashing bright red bar will be drawn if there are two cars there.
  • A few more bug fixes and usability improvements.

Version 1.4 – 4/17/2023

  • Added red/green indicators showing connection to iRacing and speech-to-text activity.
  • Added proper open file / save file dialog boxes to help with the selection of the speech service log file, background texture file, and font file.
  • A few bug fixes and stability improvements.

Version 1.3 – 4/15/2023

  • Now using Inno Installer!
  • Added a shiny new UI!
    • Editing the settings.xml file directly is no longer necessary.
  • Many more bug fixes and stability improvements.

Version 1.2 – 4/11/2023

  • Added profanity filter option, enabled by default.
  • More bug fixes and stability improvements.

Version 1.1 – 4/7/2023

  • Added support for PushToTalk to automatically mute the selected audio device when it is pushed, and to un-mute when released. This is useful if you race using loud speakers, and want to talk without having others hear those speakers.
  • Various bug fixes and stability improvements.

Version 1.0 – 4/5/2023

  • Initial release