Videogame Controller

In this demo, we'll configure Syntien to send data via BLE to an ESP32-S3 microcontroller. The ESP32 is programmed to translate BLE input into USB HID gamepad data – so any device that accepts a standard USB gamepad (Mac, Windows, Linux, Steam Deck, etc.) will see Syntien as a regular controller.

Most of the configuration in Syntien is pretty simple. We'll use two tabs for this demo. The first one is just a basic mapping of all of the buttons on a typical gamepad. We're using the

Button control to imitate the buttons of a gamepad, and an

XY Pad for the thumb sticks. All of the buttons send two bytes of data via BLE. The first token is a letter, like "A", "B", "X", or "Y". The second token is a 1 or 0; we send 1 when the button is pressed, and 0 when the button is unpressed. On the Variable Scaling screen, we set those 1s and 0s to be encoded as 8-bit integers – i.e. only a single byte of data.

The two thumb sticks are a little more involved. The host expects values from 0 to 255 representing the X and Y values, where (0, 0) is the top left corner and (255, 255) is the bottom right corner. For these controls, we send three bytes: first, a letter to indicate which button (or thumb stick) we're using. The second byte is the X value, scaled from 0 to 255, floored, and encoded as an 8-bit integer. The third byte is the Y value, scaled from 255 to 0 (because the axis is inverted), floored, and encoded as an 8-bit integer.

The second tab contains only the controls we need to play SuperTuxKart (an open-source kart racer), with a few of them expanded a bit so they're easier to interact with. We use an

XY Pad for steering and acceleration, and we use the

IMU control with some scaling to activate the skid feature; whenever the device is rotated by more than 0.5 radians, it triggers the skid button. We also use the

Gesture control to fire items. Whenever you shake the device, it sends a signal to press the fire button, and we use a second BLE output with a 0.5 second delay to un-press it shortly thereafter. Finally, we use a

Button to call the rescue bird.

Working with ESP32

If you try this yourself, you'll have to flash your ESP32 with the attached code. There are a lot of great online resources that cover that, so we won't dive into it here. You can do this using something like Arduino IDE or the PlatformIO extension within VS Code.

After you plug the ESP32 into your computer, it should advertise itself as a BLE device, and you should be able to connect to it from the I/O screen in Syntien.

BLE Characteristics

You'll see in the ESP32 code that we're advertising the BLE characteristic as "write without response" so we can send data to it without any noticeable lag.

Now, you'll be able to use Syntien 2 as a Bluetooth gamepad!

Loading video…

Download Syntien interface and ESP32 code

Extensions

There are a ton of possibilities for extending this setup.

You could configure a button to send many BLE messages in quick succession by setting up multiple outputs with the "Delay" parameter, each successively longer than the last. In other words, button mashing.
You could connect multiple devices running Syntien 2 to the same ESP32 and play a game with 2 or more players controlling the same character.
You could use the Level Meter control as a "button", and configure the scaling such that the button is only pressed when you scream loudly. Ex: add a number like 30 to the output, clip it to a maximum of 1 and a minimum of 0, then round it. Any input over -29.5 dB would be registered as a button press, and any input under -30.5 dB would be registered as a button release.

Keyboard with Stylus