Emoji Keypad

I was inspired by Dustin Watts' Bluetooth ESP32 TFT + Touch Macro Keyboard. I liked this project because it was simple enough for a beginner build, and it involved the application of a variety of engineering disciplines including coding, electronics design, and even 3D printing. I like to explore content creating, so a Bluetooth keypad that mimicked Elgato's touch stream deck would make my workflow easier. Throughout the process, I ended up modifying the website shortcuts to emoji shortcuts. I love using emojis because I think they can lighten up and make any conversation fun. My mac doesn't have a touch bar so it's hard for me to use emojis while I type on my computer. I made this project in Ms. Berbawy's Principles of Engineering class, with a lot of support from her! I hope you enjoy making the project as much as I did! Special thanks to Berbawy Makers, Autodesk Fusion 360, Arduino IDE, GitHub, and Fremont Unified School District, and Mission Valley ROP for supporting me in this project!

Materials

• ESP32 DevKitC-VE Development Board
• TFT Touch Screen
• Female - Female Jumper Wires
• Velcro
• Solder
• M3 Screws (5 mm length)
• M3 Lock nuts

Equipment

• Soldering Iron
• Magnifying glass and Helping hands (or some kind of stand that helps you hold wires in place)
• Soldering fan
• Desoldering pump
• Heat Gun
• 3D Printer (I used a Prusa MK3S+) and PLA filament
• Dial Calipers
• Exacto Knife
• Wire strippers

Applications:

• Fusion 360
• Prusa Slicer (or any other slicer)
• Arduino IDE

The first thing I did was connect the ESP32 microcontroller to the touchscreen. Both the software components have header pins on them to form connections easily. If you read the chart below carefully, pin 18 on the Esp32 connects to SCK and T_CLK on the touch screen. Likewise, pin 23 on the ESP32 connects to SDI(MOSI) and T_DIN on the touchscreen. This was the first setback I ran into. I didn't know how to connect two different pins on the touchscreen to the same pin on the microcontroller. After considering many different options, like a printed circuit board or soldering wires directly between the two parts, I decided to use jumper wires because of their portability and ease of use. Instead of ordering double-headed jumper wires, I decided to solder them myself. I will explain how to do this in the next step. For now, you should connect the other headers with female-female jumper wires.

Note: The Microcontroller has two "GND"s and you can connect it to either, it doesn't matter. Make sure to double check the connections as they are an integral part of making the software work.

These are the connections you need to make from the ESP32 -> Touchscreen:

The microcontroller pins are on the left, and the touch screen pins are on the right.

Esp32 - TFT

GND - GND

3.3V - VCC

15 - CS

4 - RESET

2 - DC/RS

23 - SDI(MOSI) and T_DIN

18 - SCK and T_CLK

32 - LED

21 - T_CS

19 - T_DO

27 - T_IRQ

I started with cutting a normal female-female jumper wire through the middle with an Exacto knife. Then, I used wire strippers to take off about an inch of the the insulation. I took another whole jumper wire, and used the knife to carefully scrape off the insulation down the middle. I accidentally cut the wire down the middle too, but then realized I could join it back with solder. Then, taking my two pieces, I soldered the wires together. I put each half on one clips on the soldering helping hands.

It is important to spread the solder evenly the first time you apply it on the wires. This is because bringing the hot iron near the wires with solder on them, melts the metal and you have to start all over again.

After the soldering, I put a piece of plastic insulation on the exposed wire area. I carefully used a heat gun to shrink it, and my double headed jumper wires were ready.

I used several software resources from Dustin Watts' "Bluetooth Keypad" library on GitHub. A big thank you to him for his original idea and source code!

I was originally going to build a "laptop shortcuts" keypad like him, but I modified that a little bit. More on that in Step 6.

1. Start off with downloading Arduino IDE on your computer. After that is done, go to Arduino on the top bar, and go to Preferences. Under that, go to the Additional Board Managers URL. 
2. Copy “and paste this link under this section’s pop-up box. Arduino Package.
3. Find the "Tools" section and select ESP32 under the board manager menu.
4. Download the "Adafruit-GFX-Library", "TFT_eSPI" and "ArduinoJson" on Arduino IDE. They are built into the app.
5. Visit these links (credit goes to Dustin Watts) and download the zip files by selecting the "code" button on GitHub. (Copy and paste these links on to your search engine)

• ESP32 BLE Keyboard
• ESPAsyncWebServer
• AsyncTCP

Now, add these libraries to the Arduino IDE workspace by going to Sketch -> Include Library -> Add .ZIP library.

After this, you need to upload the SPIFFS Esp32 flash memory to store the images that are used in this project. To do this, you'll need the ESP32 Sketch Data Upload tool. Visit this link on GitHub and "https://github.com/me-no-dev/arduino-esp32fs-plugin" and follow the instructions on the site to download it.

Visit Dustin Watts's Source Code repository for the bluetooth keypad on this link. Download the zip file and upload it onto the Arduino IDE workspace.

Now, upload the code onto the ESP32 microcontroller. Ensure that it is plugged in and connected to your computer. Click on the "upload sketch" button on the top left corner of the Arduino app and wait for the data to upload. Be patient as this may take some time.

The software I used for 3-D design was Fusion360. If you are a complete beginner, recommend that you watch basic CAD Autodesk YouTube tutorials by the channel LearnItDoItMakeIt . Secondly, take the measurements for the screen using calipers. There are two parts to the case. One part will attach to the actual screen. The second part will be a hollowed out rectangular prism that will hold the wires and the microcontroller. In the hollowed out prism, there will also be an opening for the power cable to connect to the microcontroller.

The first part of the case will have holes which are aligned with the positions of the mounting holes on the touchscreen. You have to put in screws and lock them in place for the screen to stay attached to the case.

To start the CAD process, create a rectangle that is the same size as the entire touch screen (including the red part). Then, extrude it to an appropriate height (to ensure that all the jumper wires and microcontroller can fit comfortably). You can either have a thicker touch screen holder, or a deeper triangular prism. It is up to you. I made my touch screen holder thicker, with a depth of around 30 mm. After that, measure the dimensions of just the black "touch" area. On the x-y plane of the first prism, create a rectangle of the same size. Extrude it downwards, but not all the way -- leaving a ledge for the red part to rest on. The last step is to create holes that are aligned with the touch screen's mounting holes. I used the hole function, and threaded them for M3 screws. Extrude the holes all the way through, as we will use a lock nut in the last step to hold the components in place.

For the second part, start with a right-angled triangle and extrude it horizontally to the same length as the rectangular prism you made above. Then, create 2 offset planes inside the prism. The distance between the two offset planes and the edges should be the same as the shorter width of the top case. On the first of planes, create a smaller triangle, and extrude it all the way to the second plane. The last step is making a hole in the bottom case for the USB.

Here are the things I would pay more attention to:

1. Getting the radius of the mounting holes was slightly tricky, as even a millimeter can make a difference. I searched up the radius of a M3 screw (it was the best fit through the touch screen hole) and used that in 3-D diagram.
2. It is difficult to exactly position the mounting holes for the case, and it took me a few test prints to get it right. I'd suggest that you print a scaled screenshot of the diagram, and set up physical coordinates on the paper for the holes, and translate that to Fusion 360.
3. The holes are not aligned evenly on both the sides of the touchscreen, so create some kind of marking on your case that will help you distinguish between left and right. For example, you could fillet one side.
4. When 3-D printing the second part, I needed supports (3D printers build in layers from bottom to up, so the overhang needs to sit on something in order to print). So that the supports wouldn't get stuck in the tiny pieces, I filled them up as shown in 5th picture.

Start with exporting your model to a STL file on Fusion 360. Install a slicer software like Prusa Slicer on your device if you haven’t already. Convert the 3D model into a set of instructions (G-code) that the printer can understand. Here, you can configure the settings such as layer height, infill density, and print speed based on your requirements. Ensure that you have the correct temperature for the bed. Now, import the sliced g-code onto a flash drive, and plug it into your 3-D printer. Before printing, make sure the build plate is attached properly, and it is clean. Check the nozzle settings, and its distance from the build plate. Lastly, check that the filament is installed properly. The color of the case is up to you, be creative! Start the print, and monitor it for at least 15-20 minutes to make sure that nothing is going wrong. After the print is done, carefully remove it from the plate using a spatula or another tool. 

Since we are not building the exact same "shortcuts" keypad, you need to modify the source code that Dustin Watts has provided.

1. If you are working on a mac, download the "unicode keyboard" by going to System Preferences -> Keyboards.
2. Choose the emojis that you want on your keypad. You will need 5 emojis that depict your 5 menus. For example, one folder will lead to "happy emojis' while another one could show "nature emojis". Come up with your menus, their title emojis, and the actual emojis in the folder. Each folder will have 5 emojis.
3. Search up "----- emoji file format" onto your search engine. Visit the link for that emoji on the website, "File Format" and scroll down until you see a string of numbers and letters that begin with "d83". (They will be in parentheses next to the section of UTF - 16 Hex.
4. Create a table with each of your emojis and their unique unicodes. This will tell the arduino which emoji to print, as each unicode is like a keyboard shortcut that prints special symbols.

I will attach a screenshot of some of my modified code above, use it as a template to code it for the rest of the emojis.

To display the actual emojis, look for pngs of them online and modify them to the bmp format so that they can be uploaded onto the arduino IDE workspace.

After you are done programming the microcontroller, upload the data onto the ESP32 using the same button described above. Make sure that the ESP32 is connected to the same wifi as your computer. Your computer's bluetooth should be on, and it should be paired to the "Free Touch Deck".

Choose an appropriate length of M3 screws (should be small) and fit them into the mounting holes of the printed case. Now, slide in the touch screen into the screws (make sure it is oriented correctly) so that the black screen peeks out through the cut in the case. Now, use lock nuts to tighten the screws in place in addition to the threaded holes. Your screen should stay in place!

Apply velcro onto the edges of both the cases and fit them together. You're done! Enjoy using your emoji keypad as you text your friends, or compose creative things on your laptop!

Here is a video of me with the final product!