Hello from Portland, Oregon!
TL;DR
- We finally got a full sample of the enclosure + feet + interconnect bars. There’s a problem with reverse-tilt configurations. The factory is on it.
- We’re headed back to China in ten days to check the first parts coming out of the injection molds.
We’re in Portland through this Monday for XOXO, come say hi! We should have some stickers for you.
Shipping addresses
A few of you have written to tell us that you’ve moved and to make sure we have current addresses. While we’re always happy to hear from you, don’t worry about shipping address updates. We’ll be emailing everybody who backed us on Kickstarter (and everybody who preordered on our website) to get current addresses before shipping.
Mechanical
When we last wrote, we said that we expected to have a second, fully integrated sample of the Model 01 sent to us by the week of August 14. When the factory tested out that sample locally, they found some issues with the feet and interconnect bars and ended up deciding to update the design and build a new sample.
That sample shipped out to us on August 29 and we got it on September 2. When we opened the package, we found that they’d only sent us the wooden enclosure, the plastic bottom plates, feet and center bars. Keycaps and metal keyplates weren’t included.
The keycaps aren’t too big a deal, since we had a full set of the “final, except for one tiny tweak” keycaps from just before Jesse left Shenzhen. Without the keyplates, however, we can’t combine the plastic and wooden parts with the PCBs the factory sent us a couple weeks ago to build a full working test unit. Without a working test unit, we won’t sign off on the full, final design.
Ordinarily, this sort of a delay would mean that the schedule would have slipped dramatically, but our factory did something a little bit unorthodox in order to try to keep the schedule as close to their original estimates as possible—they started making all of the injection mold tooling weeks ago with the “near-final” design, since they were pretty sure that any required changes could be made on the fly later. Minor changes to tooling are not uncommon, but this opens them up to some risk if significant issues are found before we sign off on the tooling.
As we started to test the sample, we found a bunch of minor issues with a number of tent and tilt configurations and feet not quite touching in the right places. Most of these turn out to be traceable to a pretty simple issue: The nice round rubber bumpers on the bottom of the Model 01 are a custom part designed so that all four corners of the keyboard sit comfortably on the desk. The factory didn’t have those ready, so they made the sample with a similar pre-purchased part that’s not quite the right height. When we tried to simulate the correct design by sticking on some additional rubber feet, most of these small fit issues were resolved.
We weren’t thrilled with the quality of the rubber stuck to the flip-out feet, but the factory clarified that it was also a purchased part and not what we should expect for a final design. They said they’d send us 3D CAD of the final design this week.
The tripod mounts on the bottom of the keyboard are nuts inserted into a plastic cavity. On the prototype, they’re inserted and glued after CNC milling. On the production keyboards, they’ll be inserted during molding, which in theory should make them sturdier. We had some questions about how sturdy they’ll be in the face of… harsh treatment and the factory is going to talk with their injection tooling designers about adding additional mechanical reinforcement for the nuts.
There were a few small issues with the wood finish that we said we wouldn’t be able to accept on delivered units. The factory told us that they’d make sure the wood supplier understood the issue.
One of the plastic feet was installed backward. It was an easy mistake—there were no markings on the front or back of the feet to indicate which side was the inside and which was the outside. The factory confirmed that the feet coming out of the injection molds will have a tooling mark on one side, making it much easier for the assembly line workers to do the right thing.
And then we found the big problem with the sample. When the keyboard is “negative tilted” with the front higher in the air than the back, it’s dramatically unstable. The front foot acts like a fulcrum and the keyboard pops up off the desk when you rest your palm on the palm rest. Now, there’s an argument that you should never actually rest your palms on a palmrest, but that argument is kind of bullshit.
We didn’t catch this issue when we were testing samples in Shenzhen because we didn’t yet have an integrated sample with the wood top and the feet fitting well enough that we expected it all to work right (or we would have brought it home with us.) We’ve raised the issue with the factory, they’ve replicated it and are thinking through potential solutions (which we’re pretty sure will involve moving the front feet much, much closer to the front edge of the keyboard. We’ve also got a few ideas of our own if they don’t come up with something acceptable. If nothing else works out, we may end up being really, really happy we decided to add ¼-20 tripod mounts as attachment points on the bottom of the keyboard.
There’s also a chance that with the electronics installed, the change in center of mass will mean the issue will be significantly better, but we’re not holding our breath.
We had a call with the factory late Sunday night and ran through all of these issues, as well as a few other minor items. They’re going to build out a fully integrated sample this week, test it locally with whatever improvements for the feet they can come up with and send it over to us.
On a happier note, the slide-on center bars seem to do what they’re supposed to and support the keyboard in both flat and tented configurations.
Electrical
While we were waiting for the mechanical sample, the factory sent us sample PCBs to test. Those showed up on August 23.
The factory sent four sets of PCBS. Two of them had the APA102C LEDs we’d specified and two of them had “compatible” HOD9822 LEDs from Haodong, a Shenzhen brand. (You might note the quotation marks around the word compatible in the last sentence. We’ll get to that in a moment.)
The factory proposed the Haodong LEDs as an alternative because the APA102C LEDs need to be soldered at a relatively cool 230°C, rather than the 260°C they usually use for their reflow ovens. If you solder the APA LEDs at 260°C, they have a tendency to… cook inside and never glow again. The Haodong LEDs are somewhat hardier and can handle the higher temperatures without breaking a sweat. The Haodong LEDs speak the same wire protocol as the APA102Cs. The Haodong LEDs are also just a bit cheaper. So far, so good.
In testing, the problem comes when you start driving them at high intensity. If the APA LEDs are a little bit starved for power, they get dimmer. If the Haodong LEDs are a bit starved for power, their color changes a bit. If the APA LEDs are really starved for power, they get dimmer. If the Haodong LEDs are really starved for power, they start to behave somewhat erratically.
So, as of now, we’re saying no to the cheaper, easier to solder LEDs.
Since last month, we’ve also been tracking one other issue with the PCBs. We’re using PCA9614 controllers to talk across the cable between the two hands, via the I2C protocol. They work great. However, when we were debugging an unrelated using a FLIR heat camera, we discovered that the PCA9614 chips were heating up to about 50°C. They’re not ordinarily supposed to get that hot, so we reached out to NXP, who make the chip. Our NXP sales person connected us with a field applications engineer (FAE).
FAEs are essentially the most technical of technical support. The NXP FAE vetted our schematics and, after building a clone of the relevant circuit on a breadboard, agreed that the design looked ok. The next step was to evaluate an actual set of boards, so we drove down to San Jose with a set of boards and walked through the issue with our FAE. He still didn’t see anything wrong with our design, so he asked if he could borrow the boards to test in their lab. As of last Friday, he’d confirmed that the design of the boards looks ok and that they’re seeing the heat issue with our boards in their lab. They say that this week, they’ve got someone on the chip design staff looking at the issue.
According to the datasheet, the PCA9614 chip’s temperature is within its specced operational range, but it makes us uncomfortable that it’s getting so warm. Our worry is that if the chip is giving off this much heat, it might be slowly roasting itself. This might ultimately turn out to be nothing, but we’re not sure it’s nothing, so we’d rather have NXP check it out while we can still fix it.
Other than that, the electronics seem to be behaving themselves. We noticed that we’d accidentally printed some text on one of the boards backwards. So we’ll get that fixed up before production.
Firmware
Nearly all modern electronics have firmware, essentially programs that control how they behave. These days, most of that firmware is programmed into devices before they leave the factory, never to be updated or improved.
Sometimes, that firmware is programmed/flashed onto devices after they’ve been partially assembled. Sometimes, it’s flashed directly onto the device’s microcontroller chips by a chip manufacturer or distributor before they even arrive at the factory. There are a couple reasons you might do this. The first one is that your firmware might be a trade secret and you might not want have your assembly factory to be able to make your product except with the exact chips you send them. Needless to say, this doesn’t apply to us. The second reason you might send pre-flashed chips to your factory is that it can be faster and cheaper to program them in advance.
Our factory would strongly prefer that our chips get flashed by the distributor before assembly. For us, the only real downside to this is an earlier and firmer deadline for the shipping version of the firmware. Unlike most manufacturers, we _want_ you to be able to reflash your keyboard (to configure custom layouts, build neat add-ons, change the LED effects, etc). That said, we want to make sure you can’t accidentally “brick” your keyboard by flashing buggy firmware onto it. Luckily, chip designers are well aware of this kind of issue and field-reprogrammable chips often have space on them for two completely different programs: the firmware and the “bootloader.” The bootloader is responsible for either running the main firmware or letting you replace the main program with a new version. We’ve been focusing our ATMega32U4 programming efforts on making sure that our bootloader is robust and reliable. The keyboard will, of course, ship with working firmware, but by the time it reaches your hands, there will almost certainly be better-working, more amazing firmware available to update to.
Early on, we decided to make the Model 01 “Arduino-compatible”, so we chose to base our bootloader on the same “Caterina” bootloader used by the Arduino Leonardo. Caterina is designed to make it relatively foolproof to update the firmware on an Arduino over USB without having to tap a reset button on the device or connect a specialized programmer to your board. The only thing Caterina won’t let you do is replace your bootloader with a new one.
For a development board like an Arduino, this is a reasonable compromise between security and convenience. For your keyboard, it’s another story. While we know it’s not terribly likely, it’s important to us that malware won’t have an easy time reprogramming your keyboard. When we designed the Model 01, we connected a pair of pins on the ATMega32U4 that’s your keyboard’s brain to the top left key on your keyboard. We’ve modified the Caterina bootloader not to go into programming mode when it’s reset by software unless you’re holding down that key. We’ve also updated the bootloader to be able to use the keyboard’s LEDs to tell you what’s going on.
You can find source code for our version of Caterina here:https://github.com/keyboardio/Model01Bootloader
When you want to update the firmware on your Model 01, you’ll be able to use the free Arduino IDE, the avrdude command-line tool or a little custom GUI tool we’re building. If you decide you want to replace the bootloader on your Model 01, you’ll need to unscrew your keyboard and connect a hardware programmer (or an Arduino pretending to be a hardware programmer) to the ATMega32U4’s “ICSP” header.
While the ATMega32U4 is the Model 01’s brains, another pair of Atmel chips act as its, uh, nervous system. A pair of ATTiny88 microcontrollers are responsible for scanning the keyboard’s keys and telling the LEDs what color to turn. They talk to the ATMega using the I2C protocol. The ATTiny chips don’t have a special protected part of memory where a bootloader is supposed to live. Typically, they’re used in embedded applications where you’re tight on space and probably won’t be updating their firmware often, if at all. Because of this, we’ve been working hard to make the firmware on the ATTinies robust and configurable. Our initial firmware hardcoded some sane defaults that we think will work well, but after a bit of reflection, we went back and added some extra features to the wire protocol that let us tweak the ATTinys’ behavior if we later decide that our sane defaults weren’t actually so sane.
You can find the ATTiny firmware here: https://github.com/keyboardio/avr_keyscanner
The ATTiny88 may not have specialized support for a bootloader in protected memory, but it does have the ability to reprogram itself. Last week, we discovered that many years ago, Atmel published a technote (AVR112) on how to build an I2C bootloader for devices that don’t have a protected bootloader memory area. This week, we’ve been working on a derivative of the AVR112 bootloader that, if we pull it off in time, will let you update the ATTiny firmware just by flashing a special programming tool onto your keyboard. The code is still…a work in progress, though you can find it here: https://github.com/keyboardio/attiny_i2c_bootloader
This is definitely the sort of fiddly embedded code that would most benefit from advice, help and criticism if you’re so inclined.
The actual initial keyboard firmware has seen only the tiniest of tweaks in the past month. You can still find it at https://github.com/keyboardio/KeyboardioFirmware
We’ve also started work on Arduino IDE hardware support for the Model 01. This is, essentially, a bundle of a few config files, all of the Keyboardio-specific Arduino libraries, a hardware definition, our bootloader and build instructions for the Model 01’s firmware. By installing this in the Arduino IDE, you’ll be able to build the Keyboardio firmware for the Model 01 and flash it just like any other Arduino sketch.
It’s very much a work in progress, but “Works On My Machine”[TM]. Right now, you install it by checking it out of git and syncing all the submodules. When it’s fit for use, we’ll publish tarballs with everything already checked out of the subrepos and built. We’ll also publish it in a format installable through the Arduino IDE’s board manager.
You can find the current version here: https://github.com/keyboardio/Arduino-Boards
Keycaps
As reported in the last backer update, we finalized the keyshapes before Jesse left Shenzhen. We’re still waiting on final pricing and MOQ for alternate keysets.
Schedule updates
The factory owed us an updated schedule spreadsheet this week, but it hasn’t shown up yet and we don’t want to hold this update any longer.
As of about 10 days ago, the first “T0” shots from the injection molds should start showing up next week. We’re still not sure exactly what’s going to happen with the reverse-tilt issue with the current design for the feet, but it sounds like a future update will talk about how one modifies an injection mold that’s not-quite-right. Our rough understanding is that it involves welding steel into the molds before remachining them.
We heard from Matias yesterday that the first 10% of our keyswitches will be delivered by the end of September, with the rest to follow soon afterward as they spin up their production lines.
Jesse’s headed back to Shenzhen for a few weeks on September 18, just after the Chinese mid-autumn festival. He’ll be working with the factory to resolve the issues we’ve talked about in this update and to check over the output of the injection molds.
There’s still plenty of work to do, but we’re getting ever closer to mass-producing your keyboards. As always, we’ll keep you updated on Twitter, in the Kickstarter comments and with future backer updates.
<3 j+k