Courtesy of SparkFun
Guide by PNP MARCUS
In this tutorial, we will install a modified version of Joycontrol, an open-source project that allows you to emulate Nintendo Switch Controllers on a Raspberry Pi over Bluetooth. The original project allows you to control your Switch using a simple command line interface, but I had some ideas for more functionality I could add, so I forked the repository and got to work.
Joycontrol-ms adds keyboard control, macro recording and playback, and integrates the SparkFun Top pHAT to make things handy. I’ll also dive into the code that makes everything work, in case you’d like to write your own custom commands.
Connect your Raspberry Pi to your Nintendo Switch as a Pro Controller
To follow along with this tutorial, you will need the following materials. You may not need everything though depending on what you have. Add it to your cart, read through the guide, and adjust the cart as necessary.
Nintendo Switch Macro Recording on the Raspberry Pi Wish List SparkFun Wish List
You Will Also Need
If you're new to Raspberry Pi, or the SparkFun Top pHAT, you may want to check out these pages before moving forward.
Please Note: This tutorial assumes you have already set up Raspbian on your Pi. If you haven't got your Pi setup, head on over to the Hookup Guide to get yourself squared away.
Guide for hooking up your Raspberry Pi 4 Model B basic, desktop, or hardware starter kit together.
If you have already set up and configured your Pi for the Top pHAT, you can (obviously) skip this step. Assembly for the Top pHAT is fairly straightforward. It plugs directly into the GPIO pins of a Raspberry Pi. Most importantly, users need to pay attention to the orientation of the pHAT and double check that the pins are lined up properly. Below are a few examples for users to follow. For my project, I'm using the Pi 4 Model B, though the project should work on most Pis with Bluetooth. Be sure to buy a header if your Pi requires it for clearance on the Top pHAT.
First, you'll want to connect a female header if needed, like so:
Next, snap on the Top pHAT, and connect your keyboard and mouse. Initially, you will also want to connect to an external display using one of the HDMI ports. Finally, connect the USBC power adapter and plug it in to an outlet.
Once your Pi boots, we can configure it for the Top pHAT and install Joycontrol-ms and its dependencies.
OS Configuration and Joycontrol-ms Installation
If you have already set up and configured your Pi for the Top pHAT, you can skip this step. These instructions are the same as OS Configuration part 1, WS2812B LED’s, 2.4” TFT Display, and OS Configuration part 2, from the SparkFun Top pHAT Hookup Guide. Because this project doesn’t use all of the features on the HAT, you can get away with only completing these setup steps if you’re in a hurry.
First, disable overscan by clicking on the Raspberry logo > Preferences > Raspberry Pi Configuration.
Next, click into the "interfaces" tab, and enable the SPI and I2C interfaces.
Now you can click ok, but don't reboot the Pi yet. We will return to the Raspberry Pi Configuration window later and reboot then.
Now we need to install the Adafruit Neopixel python package because the project utilizes the onboard addressable LEDs. Open a terminal and copy the following command into the window, and press enter.
sudo pip3 install adafruit-circuitpython-neopixel
Next, in order to use the 2.4" TFT display, we need to add the driver modules and a module configuration. In your terminal, copy and paste the following and hit enter:
sudo nano /etc/modules
When Nano opens, copy and paste the following to the end of the file:
Now, press Ctrl+X to exit, Y to accept changes, and Enter to write the changes and exit Nano. You'll then need to repeat this process for the module configuration. In your terminal, copy/paste:
sudo nano /etc/modprobe.d/fbtft.conf
Once Nano is open (this time the screen will appear mostly blank), copy and paste the following, then exit and save changes.
options fbtft_device name=fb_ili9341 gpios=reset:23,dc:24 speed=16000000 bgr=1 rotate=180 custom=1
Again, press Ctrl+X to exit, Y to accept changes, and Enter to write the changes and exit Nano.
Next, we can go ahead and install the dependencies for Joycontrol-ms, of which there are three. All you need to do here is copy/paste the following commands into your terminal.
sudo apt install python3-dbus libhidapi-hidraw0
sudo pip3 install keyboard
Once that is completed, you can cd into the directory where you'd like to install joycontrol-ms and clone the repo there. I cloned mine to the desktop.
git clone https://github.com/marcus-stevenson/joycontrol-ms.git
Then, cd into Joycontrol-ms and run:
sudo pip3 install .
Phew! Everything should be set up now. All there is left to do is to return to the Raspberry Pi Configuration window and select "boot to CLI":
NOW you can reboot your Pi. This is also a good time to disconnect the HDMI display as we will be using the Top pHAT display from here onward for convenience. I recommend fully shutting down instead of rebooting for this purpose. (if you were to replace your power cable with a battery pack, this project could be even more mobile).
How to Use Joycontrol
To run the program and connect to your Nintendo Switch, open the “Change Grip/Order” menu on the Switch, and cd into joycontrol-ms and type the following command:
Sudo python3 ./run_controller_cli.py PRO_CONTROLLER
Provided that you have set everything up properly, you should see a controller connect to your Switch.
A bunch of info will be printed out to your terminal on the Pi. This can sometimes take a moment or two, but eventually you’ll see “cmd>>” printed in in the terminal. At this point, you’re connected and ready to input commands into the CLI. first try:
Press enter and wait until the Switch responds. The first few button presses can take a moment to take hold, so be patient and wait for a response even if you see the “cmd>>” prompt. Once your first “A” command is recognized, you will return to the Controllers menu. If you send too many “A” commands, you will immediately re-enter the “change grip/order” screen and the Pi will be disconnected from the switch. If this happens, just use sudo python3 ./run_controller_cli.py PRO_CONTROLLER again to reconnect.
Once you’re back on the Nintendo Switch home screen, type help into the CLI and hit enter. This will show all the available commands and their descriptions, but you may not be able to read the whole thing unless you use your keyboard’s directional arrows to scroll back.
Here are all the commands:
Image Courtesy of Nintendo of America
Starting with the basics, you can emulate the press of any individual button by simply typing in the label for that button and pressing enter. For example:
You can chain multiple button presses together using “&&”:
The joysticks are less convenient to use via the CLI, but you can set them like so:
cmd>> stick left right
This command sets the left joystick to its furthest “right” position. It will stay in this state until you reset the joystick to the neutral ‘center’ position by sending the command:
cmd>> stick left center
You can “mash” a single button until you choose to stop by using the command:
cmd>> mash a 5
This presses the “a” button every 5 seconds until you hit .
While all these cli commands are fun to play around with, they don’t quite provide the functionality that I’m looking for. I went ahead and added a few more commands that allow you to control the Switch directly by binding the controls to the keyboard, to record this keyboard control, to play it back like a macro, and to delete unwanted macros. Macros persist after exiting each Joycontrol session.
First, to control your Nintendo Switch using the keyboard attached to your pi, use the command:
You will be prompted to press < enter > to bind the keyboard to the controller. To stop using the keyboard control, press < enter > again. The Keybinding is as follows:
To record a macro, use the command:
You will be prompted to type in the name of the recording, so you can replay it later. After you press enter, Joycontrol will begin recording the input you enter as you control the switch using the keyboard control (the keybinding is the same as the “keyboard” command). To stop recording, press again. While you’re recording, the RGB leds on the Top pHAT will light up red.
To play back a recorded macro, use the command:
After you send the playback command, you will be prompted to type in the name of the macro you want to replay. The saved macros (if there are any) will be listed. While you are playing back recorded macros, the leds on the HAT should be green.
There should be one macro already present named ‘mario’. This macro beats world 1-1 of the original Super Mario Bros game. If you have the NES virtual console installed on your switch, you can try out this macro yourself by navigating to the NES, picking Super Mario Bros, and pressing “plus” to start the game. Once you’ve started world 1-1, use the playback command and select ‘mario’. You should see Mario traverse the iconic first level of the game. The timing can be tricky, though. If you move Mario before starting the macro, the playback will be mis-timed and mario will die or get stuck in a corner until the macro finishes playing.
Finally, you can delete macros you don’t want any more with the command:
You will be prompted to type in the name of the macro you want to delete. While the delete_rec prompt is open, the leds on the HAT should light up blue.
In the next section, we will further explore how we can use the various features of Joycontrol-ms to "play" Animal Crossing: New Horizons.
JoyControl and Animal Crossing
Check out that Arduino!
Though I initially discovered Joycontrol while searching for a way to 'spoof' Amiibos (which you won't find details on here, but there's plenty of info out there), I quickly realized the potential for "automating" away some of the more tedious parts of the game.
I suppose I should back up for anyone who's following this tutorial for purposes other than to automate their ACNH island. If you're not familiar, Animal Crossing is a series of games by Nintendo wherein you live a virtual life as a virtual villager among your anthropomorphic animal friends (and enemies, if you play that way). Players earn bells (money) for various tasks like catching and selling insects or fish. The interface for interacting with all the various characters in this game is an endless series of text menus, many of which can become maddeningly repetitive the more you play. So repetitive, that I often make unintended selections out of sheer impatience and button mashing.
I enjoy this game immensely, I swear.
One of the things I find myself doing often is filling my pockets completely and then selling the whole contents, save my tools, to Timmy and Tommy. This is the perfect candidate for a macro. To record my macro, I make sure my pockets are full (it doesn't matter what's there, I split up a stack of wood to use as placeholders while recording) and go talk to Timmy and Tommy. You should also take note of where the gloved cursor is within your inventory, this will be important later. I make sure mine is in the top left corner.
Once I've got the menu pulled up, I start recording on my Pi.
After entering a name for your macro, carefully use the Keyboard on your Pi to navigate through the selling process exactly as you want it to be replayed. I recorded this macro to select and sell all the items in the bottom three rows of my pockets.
Hit enter once you're done recording. To replay the macro, use the playback command and enter the name you chose. Before starting playback, make sure your pockets are full, the gloved cursor is where it started when you began recording the macro, and that you've pulled up Timmy and Tommy's initial menu.
(enter the name of your macro)
If all the conditions are the same as when you recorded the macro, you should now be able to re-use it to "skip" through these types of repetitive tasks. How you use the macro recording will depend on your needs/habits in the game, but here are some of the things I've used it for:
Given the game's wide array of activities, I'm sure there's even more that I haven't thought of yet, and this tutorial barely scratches the surface as fare as using Joycontrol-ms with other games. Just don't spend all your bells at once! (or do...)
As detailed in the previous section, unwanted macros can be deleted with:
(enter name of macro to delete)
And when you're done with Joycontrol, disconnect with the exit command:
If your product is not working as you expected or you need technical assistance or information, head on over to the SparkFun Technical Assistance page for some initial troubleshooting.
If you don't find what you need there, the SparkFun Forums are a great place to find and ask for help. If this is your first visit, you'll need to create a Forum Account to search product forums and post questions.
Resources and Going Further
SparkFun has a whole range of Raspberry Pi projects and tutorials. Check out some of the links below!