How difficult was it to nail the feel of the game, specifically when it came to making sure that a player’s actions and the world–even in the cutscenes–sync up with the music?Johanas:
The general gameplay mechanics revolved around the idea that regardless of the input timing, we would interpolate the action to eventually execute on the beat. However, there were a few exceptions including when players jump, dash, parry and other general movement since those felt too restrictive. But any animation that had what we call a “lead in” would adjust its animation timing to make sure the finishing point of the action hit on the beat.
The next step was encouraging players to press the button on the beat. From a game design perspective, we made sure to not fall into a trap of creating a fail-state, where the player refuses to perform an action for not playing in time. The inputs on rhythm would reward the player rather than penalize them.
This concept alone required a lot of iteration, but the whole concept affected our pipeline for every aspect of the level and game design as well.
To make sure that events would always sync up to the beat, we would need to think backwards. Cutscenes and events would not only need to start on rhythm but usually at the top of the measure, so we needed to have the player perform certain actions and sometimes interpolate them to take a bit more time so a transitional sequence would always trigger at the right time. Or we’d create alternate musical transitions depending on how much time was left until the first beat, so it didn’t feel too awkward.
All cutscenes were created in musical time with a metronome starting at the first beat of a measure and always ending at the end of a measure so we can smoothly transition to either gameplay or other aspects seamlessly both visually and musically.
It required an insane amount of coordination and planning out everything, but also the assistance from systems our programmers created to ensure that everything tied into musical cues, the rhythm of the song, or user inputs.
Nakamura: Most of the team members outside of the Director and Sound team were struggling to understand basic music logic since they didn’t have much experience with music. To help, we implemented a simple UI for debugging use in-game.
It was a simple UI like sheet music, constantly showing a 4/4 beat where we could see when a button was pressed or when a gimmick was activated. Using this debug display, we could identify things like where we needed to adjust animation timing, or when things didn’t sync up with the music.
Were there any particular Unreal Engine tools that made this process easier?
Nakamura: The animation editor made creating all the player and enemy animations very convenient.
The programmers could adjust certain playback speeds, immediately check them in-game and then give concrete directions on how to adjust the motion data. This was an efficient workflow which reduced the back and forth between requesting animation fixes.
When creating the environments, we used the Editor Utility feature, which allowed us to change the parameters of all objects in the level with a single command when there would be changes to the design.
What made you decide to go with cel-shading and how was that look achieved?
Johanas: The cel-shading look was a direction from the very beginning. While the style hadn’t been determined in the pitch, we wanted something that visually was experimental but also nostalgic of a late 1990s and early 2000s era where games took a turn to creatively push art styles in games. While there have been multiple cel-shaded games, I pushed for something that felt simple but polished. I asked for no color gradations, and nothing that would feel 3D.
Our original concept art was a 2D image of the first stage with its blue sky and exaggerated architecture. I took it to the team and requested that we make the visuals indistinguishable from that 2D art.
Internally, we focused on three keywords that helped define our style, which were “Colorful,” “Sharp,” and “Clean.” They were simple enough to make a clear identity without too much detail. Over time, we layered on different techniques, but surprisingly, it was always adding to the initial look we established without redoing or reapproaching our goal.
Kosuke Tanaka, Hi-Fi RUSH Lead Graphics Programmer: Achieving our cel-shading look was a challenge and involved a lot of experimental back and forth between the graphics team and the various art teams. Graphic techniques have evolved since the late 1990s and early 2000s, and in order to give a newer take on cel-shading, we felt it was important to try to incorporate modern 3D lighting techniques such as deferred lights, volumetric fog, ambient occlusion, improved shadows, global illumination, real-time reflection, bloom, and shader lens flares in the expression of our own colorful 2D style.
Unreal Engine provided us with a great base to work from. Our ambient occlusion, real-time reflection, dynamic shadows, static global illumination and bloom are stylized extensions of the base Unreal Engine functionality. Engine modifications were necessary to achieve our look, but we were careful to make changes to the base engine code. A lot of our modifications are contained in original additional passes, making engine merge conflicts less troublesome.
A clean, sharp image was an early art goal, and this meant aiming for maximum native resolution on our target platforms. For consoles, this means 1440p on Xbox Series S and 4K on Xbox Series X. A goal set with a clear warning that since our game is a rhythm action game, a solid 60 FPS takes priority over graphics. We were disciplined in our core tech selection and always added features with performance in mind. The graphics and all art teams are happy and proud that we were able to achieve a stylized look while making sure the game runs great.
Source: Unreal Engine Blog