Rocksmith+ Dev Diary – February, 2022: Audio Updates

During the Rocksmith+ beta we received invaluable feedback from you, our users,
on the state of the audio and note detection side of the Rocksmith experience.
Some of this feedback was positive, some was critical, and we made sure to take
all of it to heart. We pored through the Discord and forum going through your
feedback. The direct interaction we were able to have with many of you pointed
us not only songs, but specific phrases, passages, or chords that were proving
to be problematic.

To all of you participated and helped us with your feedback I want to take a
moment to say thank you. It is your passion for this product that drives us
forward and helps to shape the experience, we couldn’t do it without you. As
such, we have continued moving forward with the things you enjoyed, and we are
working hard to improve the issues that were surfaced. The community has
directly impacted our priorities and helped us to home in on the more subjective
aspects of gameplay related to note detection as well as our newly added “other
interface” support and it is on those subjects that I would like to take a
deeper dive today.

The subject of chords is a really interesting one. We have worked on them a lot
(please don’t tell production how much) to make significant improvements over
how chords were detected in Rocksmith 2014. There is a fine line that we need to
walk between outright detection perfection and feel as a player. We can push the
limits of the system so that we need to hear every note cleanly ring out, but
the fact is, that isn’t really how we play guitar. To require that level of
precision every time not only feels robotic as a player but sounds robotic too!
So, while it is our goal to help guide you on your musical journey, be that
teaching you from the beginning or helping you hone the skills you already have,
we don’t want to lose sight of the fact that music is a performative art.
Detection needs to feel good and feel accurate while encouraging you to perform
and have fun, and therein lies the challenge.

This is where we have spent countless hours, staring at wiggly lines that are a visual representation of the instruments’ output. From here we can adjust a multitude of parameters to better reflect what has been played in the results that we get. Building out a robust tool set was a huge leap forward for improving note detection and being able to both see and quantify what detection is hearing.

Our goal in development was to take detection of chords as far as we could, to
make it as accurate as possible so we knew what we could do and walk it back
from there. As many of you experienced and commented on in the beta chords were
strict. We’d walked it back a bit from the zenith of outright perfection, but
many of you made it clear that we could loosen things up a bit more to achieve
our goal of keeping the experience organic, one that encourages musicality over
robotic perfection. So, how do we maintain accuracy while loosening up the feel?
Through a lot of work to be completely honest. You were able to give us reports
pointing to phrases and specific chords that were problematic and we went
through and played all those songs and hammered away at those chords. Through
examining all this data we were able to determine patterns about what was and
wasn’t working.

With this data in hand we able to go back and fine tune how we were picking
notes in chords and the weight that those notes carry. In Rocksmith+ if we ask
you to play a major chord and you play it minor; we know. This was something we
struggled with in 2014, and an improvement we remained committed to keeping.
Similarly, if there is a seventh in a chord, we are serious about that color
tone. Often times, on guitar, a chord will have a series of unique notes as well
as some that are doubled. A six-string major bar chord, for example, has three
root notes, two fifths, and only one third, but it is that single third that
really defines the quality of the chord. This third, along with the root note in
the bass of the chord, we weigh the importance of hearing pretty heavily as they
are the most defining features of that chord’s harmonic function. There is much
more going on behind the curtain, but those are some of the factors we consider
in balancing musicality with accuracy.

We also did a complete reevaluation of vibrato detection based on the feedback
we received. We realize that vibrato is an expressive tool in your toolbox as a
musician and to judge you harshly on how you are expressing yourself in the
moment is not what Rocksmith+ is here to do. We were looking for something very
specific in this first pass at vibrato detection and many of you were very vocal
about not feeling like detection was hearing you not enjoying the experience of
playing vibratos. We have redesigned the system from the ground up so you can
now choose to be expressive in the way that fits your personal musicality. We
hope that you enjoy it, and if you don’t, tell us! We’re listening.

Now, let’s talk a little bit about a new feature we added to Rocksmith+ “other
interface” support. Supporting other interfaces on PC is a big change for us and
a decision that we did not take lightly. Continuing to only support the Real
Tone Cable would have been, by far, the simplest solution for us as it removes a
variable that we have to develop around. We know there are a lot of places in
the world where getting a Real Tone Cable falls somewhere on the spectrum
between expensive and impossible, while other players already have their own
hardware that they would prefer to use. We want to welcome all of you into
Rocksmith+ in an official capacity with no third-party solutions necessary.

For a lot of you I know the excitement here is going to be ASIO support. In
Rocksmith+ we have implemented native ASIO support, not ASIO support through any
sort of Windows audio/Wasapi translator. Now there are some positives and
negatives to this approach. The biggest positive is the low latency that ASIO
provides, the immediacy between hitting a note on your instrument and hearing it
play back that is so immersive and just feels right as a player. While we have
worked hard to optimize the experience with the Real Tone Cable there will
always be operating system overhead that you just can’t get around without
integrating ASIO support. The potential downside to ASIO is that it is, by
design, full duplex. Full duplex means that your input and your output are
coming from a single device, it is even right there in the name, ASIO stands for
Audio Stream Input/Output. Being sure to route all your audio through a single
device is part of how ASIO achieves such low latency, so you have to decide for
yourself if that is a sacrifice that makes sense for your playing experience.

Other interface support is not just limited to ASIO devices, it also works with
Wasapi which is the Windows Audio Session API. Wasapi is Microsoft’s low latency
audio solution for Windows. Wasapi brings the flexibility to choose nearly any
input and output combination that ASIO is lacking, but at the sacrifice of the
extreme low latency performance that ASIO brings to the table. This doesn’t mean
that Wasapi is high latency; an optimized setup can hit a round trip latency of
about 25 milliseconds. Round-trip latency is how long it takes from the time you
pluck a note on your instrument to when that sound comes out of your speakers.
To put this in perspective this is about the same as standing roughly 25 feet
from your amplifier, readily achieved on a good-sized stage, so this is an
amount of latency that you could easily experience in the real world with a
completely analog setup. Wasapi support means that you can bring a different
cable, if that is what is available to you, or you can even play Rocksmith+ with
a USB microphone and an upright bass like the dev team has done. The Real Tone
Cable is also a Wasapi device. There are a lot of options out there in the world
so be sure that any hardware you want to use with Rocksmith+ supports operation
at 16 bits and 48 khz.

For the beta this feature came in hot and while it worked perfectly for some
others got connection errors, which is a frustrating experience. The team has
put in an absolute ton of work to make this system more stable and reliable than
what we were able to show in the beta last July. The engineers working on this
feature have tracked down and fixed a lot of bugs and we have gone so far as to
completely revamp the system that connects you under the hood. In addition to
stability improvements, we’ve worked on making some feature improvements as
well. Many of you noticed during the beta that you were unable to change the
buffer size of your ASIO device, and I am happy to say that we now support being
able to change this from inside Rocksmith+ itself. The other thing lacking in
the beta was a simple way to choose your input channel for those whose
instrument input was not channel one. While there was a work around for this it
was complicated and obviously not how the feature was supposed to function. We
now support changing your input channel from the settings for your device inside
of Rocksmith+.

We have made huge strides in the functionality of our other interface support,
but we do still consider this to be a beta feature at launch. While some of us
have huge stacks of interfaces on our desks grazing the ceiling there are simply
too many options out in the world for us to test them all. Our focus has been on
the most popular devices and the ones produced by our partners, and we will be
sure to provide a list of known supported devices at launch. As always, thank
you for the support, and we’ll be talking to you all again soon!

-Brian Poedy,
Lead Audio Designer