Articles on: Gaming Consoles

How to Control an Xbox Adaptive Controller with Your Face and Voice using PlayAbility Adaptive Software

This guide shows you how to use PlayAbility Adaptive Software with a Raspberry Pi Pico W or Pico 2W to control your Xbox Adaptive Controller (XAC).

The setup connects PlayAbility to the Pico via Bluetooth, while the Pico plugs directly into the XAC’s USB port — no additional adapters or dongles required.



Once configured, your camera, microphone, and peripherals all become part of a unified control system — letting you merge adaptive hardware, voice, and motion into one seamless experience for your Xbox or PC games.




1) What You’ll Need


  • Raspberry Pi Pico W or Pico 2W

(Only these versions support Bluetooth.)

  • Micro USB Cable

(Used to flash the firmware and connect the Pico to the Xbox Adaptive Controller.)

  • PlayAbility Adaptive Receiver Firmware (XAC USB version)

(Download the latest .uf2 file from the PlayAbility firmware releases.)

  • PlayAbility Adaptive Software

(Download from playability.gg.)

  • Xbox Adaptive Controller (XAC)

(Connected to your Xbox or PC with your preferred switches, joysticks, or pedals.)

  • Webcam + Microphone

(For face, head, and voice input.)

  • (Optional) Keyboard, Mouse, or Gamepad

(All can be merged into your PlayAbility profiles for hybrid control.)



2) Flash the Pico W Firmware


  1. Enter Bootloader Mode

Hold the BOOTSEL button and plug in the Pico W via USB. It will appear as a drive named RPI-RP2.

  1. Copy the Firmware

Drag and drop the .uf2 file onto the drive. The Pico will reboot and start running the firmware automatically.



3) Pair the Pico W with Your PC (Bluetooth)


  1. Power on the Pico W and hold BOOTSEL for 5 seconds until the LED blinks rapidly — this means Bluetooth pairing mode is active.
  2. On your Windows PC, open Bluetooth Settings → Add device → Bluetooth.
  3. Select PlayAbility Receiver when it appears.
  4. Once paired, the LED will stop blinking.


💡 You only need to pair once — Windows will reconnect automatically next time.



4) Connect the Pico to the Xbox Adaptive Controller


  1. Plug the Pico directly into the USB port on the front of the Xbox Adaptive Controller using your Micro USB cable.
  2. The XAC will detect it automatically as a wired controller.
  3. The Pico’s LED will blink during initialization and stay solid once connected.


⚡ This creates a bridge: your PC sends inputs over Bluetooth to the Pico, which instantly forwards them over USB to the XAC.



5) Connect PlayAbility to the Receiver


  1. Launch PlayAbility Adaptive Software on your PC.
  2. Open the Gamepad tab and click Enable Serial Output.
  3. A Connect to Receiver button will appear — click it to link PlayAbility with your Pico.
  4. Once connected, PlayAbility will send virtual gamepad inputs through Bluetooth to the Pico, which translates them into USB controller signals for the XAC.



6) Map Your Inputs


PlayAbility can turn any input — camera, mic, keyboard, mouse, or external gamepad — into Xbox controls.


Input

Type

Xbox Action

Smile

Face Gesture

A Button (Jump)

Turn Head Left/Right

Head Movement

Left Stick (Move / Steer)

Say “Fire”

Voice Command

X Button (Shoot)

Say “Pause”

Voice Command

Start Button (Menu)

Move Mouse

Mouse Motion

Right Stick (Aim)

Press W

Keyboard Key

Up / Forward

Left Stick (Physical Gamepad)

Gamepad

Left Stick (Move)


You can customize these mappings anytime in the Profile Editor to match your preferred gestures and layout.



7) Combine with Your Xbox Adaptive Controller


PlayAbility is designed to complement the XAC, not replace it. You can merge digital and physical controls for ultimate flexibility.


Input Type

Device

Action

Head Motion (Yaw)

PlayAbility

Move / Steer

Smile

PlayAbility

Jump (A Button)

Voice Command “Fire”

PlayAbility

Shoot (X Button)

Sip/Puff Switch

XAC

Fire (Right Trigger)

Mouse Movement

PlayAbility

Aim (Right Stick)


This hybrid setup lets you mix gestures, speech, and traditional hardware for both comfort and precision.



Troubleshooting


XAC Doesn’t Detect the Pico?

Try another USB port or confirm you flashed the XAC USB firmware version.


No Inputs Detected?

Make sure Toggle Output is enabled in PlayAbility and that your active profile includes the inputs you’re using.


Bluetooth Won’t Pair?

Ensure the LED is blinking (pairing mode). Remove any old pairings from Windows, then retry.


Firmware Update?

Simply repeat the flashing process from Step 2 with the latest .uf2 file.



⚠️ Note on XAC Trigger Outputs


The Xbox Adaptive Controller (XAC) cannot receive direct trigger output signals (LT/RT) from external HID devices such as the PlayAbility Receiver.

This is a limitation of the XAC’s USB HID protocol.


To work around this:


  • Assign an alternative button in PlayAbility (for example, A, B, or RB) to act as LT or RT.
  • Then use the Xbox Accessories app to remap that button to LT or RT within your XAC configuration.
  • You can also use connected physical switches or analog triggers directly if preferred.



9) Play with Every Input You Have


Once connected, your Raspberry Pi Pico W becomes a universal bridge, translating all your chosen inputs — from your face, head, and voice, to keyboard, mouse, and gamepad — directly into Xbox Adaptive Controller actions.


You can:


  • Steer or aim with your head or mouse.
  • Jump or attack with a smile or key press.
  • Trigger actions by voice.
  • Merge every input type into one perfect setup.


Together, they create a flexible, fully adaptive control system — no extra hardware or coding required.


Tip: Start simple with a few gestures or commands, then expand to include mouse or keyboard movement for full analog control.


🎮 Your face, head, voice, keyboard, mouse, and gamepad are now all part of the controller — play your way on Xbox and PC.


Updated on: 17/11/2025