HONG
HUA

Auto~Wave~Machine

Auto Wave Machine is a machine-learning-driven guitar effects pedal. It utilizes computer vision to translate the performer's posture into a real-time tremolo sound effect. This allows the guitar's tone to synchronize with the performer's body movements, creating a medium for a more dynamic musical performance. Furthermore, this instrument serves as an exploration of the relationship between humans and algorithms in the creative process.

Introduction

Different music genres boast a variety of iconic guitar tones – the warmth of jazz, the overdrive of blues, the distortion of heavy metal, or the reverb of post-rock. Guitarists craft these distinctive tones by linking various effect pedals to modify the electrical signals from the guitar pickups. The Auto Wave Machine introduces a unique tremolo effect that produces swinging tones. In contrast to conventional pedals that rely on physical knobs for adjustments, this innovative device employs a camera to capture the guitarist’s posture and maps body angles to the width and depth values of the tremolo effect. As a result, when the guitarist sways their body while playing, the guitar tone trembles in harmony.

The Auto Wave Machine introduces a novel approach to tone control in guitar performance, departing from the traditional one-time setup to embrace a real-time and improvisational dynamic. Guitarists now have the option to either master the device, shaping the expected outcomes through choreography, or adopt a collaborative mode, allowing the tones to transform fluidly with the vibe. This innovation adds a layer of dynamism to live performances, introducing elements of uncertainty and randomness that contribute to a more dramatic musical experience.

About Technology and Music

When discussing AI in the music industry, the focus often revolves around examples of AI composing music or expediting the music production process. However, there is a tendency to underrate other aspects of music, such as the innovation of instruments and the dynamics of live performances. In the realm of live performances, the amalgamation of visual, audio, and even haptic elements creates an immersive experience for the audience, enhancing the auditory experience of the music itself.

The body postures of musicians on stage are particularly captivating, featuring movements like headbanging, guitar spinning, duckwalk, and more. These postures are intricately linked to the timbres of different genres and contribute to the diverse array of guitar tones. The Auto Wave Machine project aims to forge new connections within these factors and explore design opportunities that arise from their interplay.

Design & Development Process

Auto Wave Machine has undergone multiple iterations to optimize its design and development. In this latest version, a Rasperry Pi operates Tensorflow's MoveNet pose estimation model and manages an analog circuit through GPIO pins. The decision to apply analog circuits for tone modulation stems from the prevailing myth in music that only analog circuits can preserve the dynamics of the original electrical signal.


- Technical Schematic


The front of the device features several control interfaces, including the master volume knob, the master switch, and the tone mode selection toggle. With this dual independent hardware setup, users can integrate the Auto Wave Machine into any position in the effects chain, just like a traditional pedal. Designed with simplicity in mind, the device boasts a visual style characterized by a panel with a parametric ripple texture that mirrors its functionality. This, coupled with a combination of white and frosted transparency, gives it an aesthetically pleasing appearance.

Additionally, the Auto Wave Machine is equipped with a standard tripod connector at the bottom, facilitating easy setup in any performance scenario.



-Prototype (Front)



-Prototype (Back)



-Prototype (Inner Structures)



-Prototype 2 (CAD Sketch)

Experience and Future

Some of my guitarist friends had the opportunity to experiment with the Auto Wave Machine, and while it was generally recognized as a playful pedal, getting used to it did require some time. This version faces certain technical challenges, including a significant delay in data processing with the computer vision model and noise impact on the analog circuits due to wireless signals generated by the Raspberry Pi.

However, despite these challenges, the project marks the beginning of infinite possibilities. It prompts us to explore how posture recognition can be further connected with various effect types and tone controls, ultimately enhancing the expressive potential of live music performances.