Hello from Oakland,

We’re going to miss Christmas deliveries for most, if not all unshipped Kickstarter and pre-order keyboards. There is an outside chance that we’ll be able to ship out about 500 more keyboards before Christmas, but we’d rather not mis-set your expectations (again).

We feel horrible about this. We’re really, really sorry. Please know that we’re not slacking off. We’ve been continuing to have nightly conference calls with the factory and have been pushing the suppliers as hard as we can to try to catch up. If you want to know what’s been happening, read on.

The first wood supplier

As we wrote in the last backer update, we’re stuck waiting on the wooden enclosures for the Model 01. The initial supplier had blown through their promised date for the second thousand enclosures. They told us we should expect our 1000 sets of pristine-quality enclosures by “the middle of November.”

Well, a few weeks ago, the middle of November rolled around. Unsurprisingly, the wood supplier delivered zero sets of wooden enclosures. On the 20th, they said they’d arrive by the 25th. On the 27th, they said they’d arrive on the 28th. On the 28th, they said they were “just packing them up now” and that they’d arrive on the 29th. On the 29th, they said “some” would arrive on the 30th. On the morning of the 30th, the wood supplier said that they would deliver the enclosures that afternoon.

On the afternoon of the 30th, the wood supplier did indeed deliver some enclosures. On friday, the factory reported that they had 500 sets of wooden enclosures for the Model 01 in their warehouse. Our factory reported that having checked the first 100, about 90% of the enclosures were good quality and that only 10% of them had issues. This is dramatically better than previous deliveries which saw 40-80% defect rates. On Monday, we got a note from the factory that they had checked samples from the remaining 200 enclosures and found a 30% defect rate.

Tomorrow, the team from our factory will be visiting the original wood supplier to find out what went wrong and to make sure they really understand the gravity of the situation.

We still don’t know when the wood factory will deliver the 500 sets they still owe us from this first order. We found out on Monday that they aren’t even working on them, despite previous assurances that they were almost done. Based on previous experience, we think it may or may not be sometime before the heat-death of the universe. They did tell us they had 400-500 sets of discolored but otherwise high-quality enclosures that they want to dye with a dark stain. That number seems…implausibly high. We’ve asked for clarification, but we’re not holding our breath.

Backup plans

As we also wrote in the last update, we haven’t been sitting around waiting for the first wood supplier to deliver. From an initial list of more than 20 possible wood suppliers, we got quotations from about 10. Of those 10, we got samples made by three. Of those three, we’ve signed contracts with two and are waiting on an updated sample from the third.

Wood Factory A

The factory signed a contract with the first supplier (hereafter ‘Wood Factory A’) on November 9th. When the contract got signed, the supplier quoted a 21 day lead time for delivery of 1000 sets of enclosures. After contract signature, they delayed production of the “golden sample” by about two weeks for reasons we still don’t understand. As of this Wednesday, they had finally begun mass production of the enclosures, but said that the work was going slower than they’d planned on and that we should expect delivery of 1000 sets of enclosures on December 22.

Enclosures from this factory will be marked 'A’ somewhere on the inside. (You won’t be able to see the markings without disassembling your keyboard.)

Our factory is pushing hard on Wood Factory A to explain and resolve the slowdown and is reaching out to their management team to try to get them to devote more resources so they can deliver sooner.

We suggested that our factory ask Wood Factory A if they could deliver half of the production run at least a week earlier. They said this should be very doable.

Wood Factory B

Contract negotiations with the second supplier (hereafter 'Wood Factory B’) were a little bit more complicated. For reasons that we can’t really get into, we ended up having to contract directly with Wood Factory B, rather than having our factory act as a pass-through. This isn’t anybody’s ideal scenario, but should be workable.

Talks with this supplier almost broke down over issues related to schedule and cost. At the point where we were pretty sure it wasn’t going to work out, they took it upon themselves to FedEx overnight us the sample enclosures they’d made. They were nice. They were really nice. They were the nicest enclosures for the Model 01 we’ve ever seen.

We got the contract for Wood Factory B sorted out late on Thanksgiving night and wired a deposit to them the next day. They started preparing the materials for mass production this past Monday. Their lead-time estimate is 21 days, putting the delivery of their 1000 enclosures sometime in the December 19-21 range.

Enclosures from this factory will be marked 'B’ somewhere on the inside. (You won’t be able to see the markings without disassembling your keyboard.)

Our factory asked them if they’d be willing to deliver the first half of the order at least a week earlier and they agreed.

Wood Factory C

We’ve been continuing to work with the third supplier (hereafter 'Wood Factory C’), but have not yet signed a contract with them. We’ve been really happy with the interaction with their team, as well as their communications and the attention they’ve payed to our design. Unfortunately, the first samples they sent us weren’t at a quality standard we’d be willing to ship to a customer.

When we rejected the first samples and asked if they would try again, they told us that they were sorry they couldn’t meet our needs and withdrew their bid. We wrote to them to ask what was going on and to see if we’d done something wrong. As it turned out, the issues were that they could no longer meet our initial deadline (because it was now only about 10 days out) and that their initial quotation was based on using less expensive production techniques that…just weren’t very good.

They’ve made a new set of samples for us. We expect them to arrive on Monday or Tuesday. At this point, we don’t expect that Wood Factory C will be making enclosures for any preordered keyboards, though if their quality is good enough, we’ll likely have them make enclosures for some of the additional keyboards we’re ordering for later sale.

If we end up contracting with them, enclosures from this factory will be marked 'C’ somewhere on the inside. (You won’t be able to see the markings without disassembling your keyboard.)

More wood factories

Our factory has also had a sample made by another supplier, Wood Factory D. We don’t know what their price or lead time is (and haven’t seen the sample), but we’re hoping to get all that information soon.

We also reached out to an old friend we’ve worked with in Shenzhen previously. She’s been working with two more factories (E and F, respectively) to get quotations and have samples made. We’ve seen photos of E’s sample, though F is still working on theirs. We’re pretty sure that E won’t be workable.

MP1 Post-mortem

On the whole, it sounds like those of you who have received keyboards have been enjoying them. As of now, the fastest typing speed we’ve heard of is 110 words per minute (up from that typist’s 80 words per minute on a MacBook keyboard.) If you can beat that, definitely, let us know.

When you ship any physical product, some percentage of the units that arrive in customer hands will be defective. This is true for just about everybody. As a general rule, the more complex your product is, the higher your failure rate will be. Moving parts, in particular, are prone to failure. (Google 'Butterfly Keyboard Defect’ and you’ll see that even the biggest computer makers in the field are not immune ;)

It’s relatively uncommon for manufacturers to publicly disclose defect rates, or even to disclose defects if they don’t absolutely have to. Customers or potential customers might misinterpret it as an admission that the product wasn’t well made and it might deter future sales.

Unsurprisingly, we think about things a little bit differently. We’ve decided, at least for this first mass production run, to publicly catalog the issues that have been reported to us. It’s an important part of what anybody shipping hardware goes through and we think you might find it interesting.

NOTE: If your Model 01 has any of the issues we describe below (or some other problem), please email help@keyboard.io. Kickstarter comments and forum posts aren’t easy for us to use to track problems and it’s really important that we 1) resolve the issue for you and 2) are able to keep an accurate count of the keyboards affected by a given problem.

Key chatter

By far, the biggest issue that we’ve been working with our MP1 customers with quiet switches to fix has been key chatter. Chatter manifests as duplicate keystrokes, even when you only press a key once. For example, a chattering “f” key might result in “f” “ff” or “fff” when pressed. Key chatter is often intermittent and is caused by the electrical contacts inside a switch making and breaking an electrical circuit repeatedly in a short period of time.

Many mechanical keyswitches may chatter a little bit as they’re being pressed. Because of this, keyboard firmware often includes a feature called 'debouncing’, which ignores changes to a key’s state unless the state doesn’t change for a relatively long period of time. In the case of our firmware, the default debounce period is about 4.5ms, which is roughly the amount of time our switch manufacturer recommends.

As of now, we’ve had reports of 48 quiet-click Model 01 keyboards with key chatter. This represents nearly 5% of the quiet click keyboards we’ve shipped. That number is unacceptably high. Both we and our suppliers take this very seriously and we are all working hard to make sure it doesn’t happen in the future. We’re thankful that in most cases, the problem has been easy to resolve and that in every single case, the customer who ran into it was incredibly patient and understanding as we’ve helped them get sorted out.

In all but a handful of cases, we’ve been able to step the affected customers through an easy at-home resolution. In two cases, we’ve mailed replacement keyswitches to customers who wanted to replace the misbehaving switches at home. In about a half dozen cases, we’ve sent out replacement keyboards or replacement electronics to customers with recalcitrant switches.

It appears that in almost all cases, the problem was caused by excess lubricant applied by the keyswitch factory. As we dissected misbehaving switches, we found that the lubricant migrated from the plastic slider to the electrical contacts, trapping grit and causing unreliable connections when the affected key was pressed. We believe that the problem was exacerbated by our packaging vendor adding extra foam to the keyboard box. This extra foam pushed some keyswitches down part way during transport making it easier for dust to get inside them.

The factory has spoken with the packaging vendor and they’ve promised to correct the foam issue before the next production run.

The lubricant, which is only applied to the 'quiet’ switches, is supposed to make the switches slide more smoothly, though after investigation neither we, nor the switch designer have been able to feel any difference at all between lubricated and unlubricated switches.

We’ve developed an at-home technique that resolve this issue for almost everybody. Because it’s really, really important that we know who this is happening to, we’ve decided not to publish the technique for now.

If you’re running into this problem, please email us at help@keyboard.io and we’ll step you through the at-home solution. If that doesn’t work, we’ll get a replacement keyboard or replacement parts out to you ASAP.

The switch manufacturer has been helpful and proactive as we’ve been working through this issue. After we first diagnosed the potential cause, they had our factory send them about 2400 switches to evaluate. They took each of them apart and confirmed that the problem was that the lubricant was misapplied or overapplied. (It turns out that the lubricant is applied by hand during assembly.)

They offered to replace all unused quiet switches with newly manufactured unlubricated switches. We’ve asked them to go ahead and start manufacturing those new switches. They expect to begin deliveries in the middle of this month. Depending on on how things shake out, new switches may or may not arrive before the next batch of keyboards is manufactured.

Because almost all of the chattering switches we’ve seen can be easily repaired at home, we’ve decided NOT to delay manufacturing in the (relatively unlikely) event that we have enough enclosures before we have new keyswitches.

We have enhanced the Model 01’s built-in 'test mode’ to be very aggressive about detecting potential chatter issues and have started to train the factory to help them catch more of these issues on the assembly line.

If we end up producing more Model 01s with the old keyswitches and one of them is earmarked for you, we will contact you before it ships and give you the option of waiting for a keyboard with the new keyswitches.

Loose USB cable connections

Seven customers have reported loose-fitting USB cables. This is due to a CNC milling error by the wood factory. They didn’t make the cutout for the USB jack quite big enough, which causes the wood to push the plastic cover for the USB jack out of alignment.

There are a few potential solutions for this issue. The simplest is to slightly loosen the bottom screws nearest the USB jack. In most cases, this relieves the pressure on the plastic USB jack cover and fixes the alignment. A slightly more advanced solution is to remove the left hand wooden enclosure and lightly sand or file down edge of the wood near the USB jack.

If your USB Type C cable feels loose and these solutions don’t work for you (or you aren’t comfortable trying them), please email us at help@keyboard.io and we’ll figure out a solution that works for you.

Going forward, all the wood suppliers have been advised to verify the dimensions on the USB jack cutout of each enclosure they produce. At the same time, the factory’s QC team has updated their procedures to reject any keyboard that shows an out of alignment USB jack.

Cracked or warped enclosures

So far, we’ve seen seven cracked wooden enclosures and 3-4 warped enclosures. Two enclosures were cracked coming out of the box. The rest cracked after several weeks of use. (As yet, nobody has told us about an enclosure that cracked due to a drop, though we’re sure it must have happened.)

A few customers have reported that one half of their keyboard doesn’t sit flat on a table. In the cases we’ve seen, this appears to be caused by the wooden enclosure warping and pulling the baseplate out of true.

If your Model 01’s enclosure has cracked or warped, please email us. We have some replacements available, though we may recommend that you wait until after we have enclosures from the new batches on hand. If your enclosure cracked due to accidental or intentional damage, we may still be able to help you out. (We have a few unsaleable enclosures that may be better than what you have.)

These problems are something we expected and, at least as of now, the failure rate we’ve been seeing is not a huge worry. Some of the issues are likely caused by wood that hadn’t been dried well enough before being milled, though some of it is just what happens when working with wood.

The factory has advised all of the wood suppliers of these issues and they’ve all vowed to be careful about drying and shrinkage going forward.

Warped stands

Somewhere on the order of a dozen customers have reported to us that one or both of their keyboard stands aren’t perfectly flat on the bottom side. This is likely due to a slight misconfiguration of the injection molding machines used for producing the stands.

The factory has talked to the supplier who makes the stands and they’ve said they’ll pay more attention as they produce the stands going forward. At the same time, the factory’s QC team will check each stand on a sheet of glass (or some other perfectly flat equivalent) to make sure that the supplier is doing what they said they would.

If your stands seem to wobble on their own (without the keyboard attached), please email us so we have an accurate count of who needs replacements once we have them available.

USB Cables

Three customers have reported that the USB cables we’ve shipped them didn’t work at all or didn’t work with the Type C connector in a particular orientation. As far as we can tell, these were all due to defects in the cables. The factory has said that they’ll push the cable supplier to do more thorough testing of each and every cable (which they were already supposed to be doing.)

If the cable that came with your Model 01 is wonky, please email us. We don’t yet have replacements from the factory, but can figure something out for you (and would like to be able to report accurate numbers to them.)

Baseplate stand mounts and stand screws

We’ve had two reports of baseplates without screw threads on the threaded inserts for the stands. This is due to poor quality control at the factory that makes those inserts.

Our factory has pushed the supplier to pay a bit more attention to quality control and added explicit checks for those screw threads to their own quality check procedure.

At the same time, we’ve seen a few of the black screws used to attach the stands to the keyboard with…less than ideal milling or finishing. The factory says they’ve switched suppliers for this part going forward and will keep a close eye on the quality during assembly and test.

DOA Keyboards

Two almost completely non-functional keyboards were delivered to customers. In both cases, we believe that the issue was caused by a MIC2019-YM6 current limiting chip that, for some as-yet-undetermined reason, failed.

Since each and every keyboard went through a 72 hour burn-in test before leaving the factory, we don’t know what happened. Nobody has, as yet, reported that a working keyboard has failed in a way that would indicate a problem with the MIC2019-YMC.

We’ve advised the factory of the issue, but are somewhat stumped about what went wrong here.

While we’d expect that anyone whose keyboard was DOA would have already contacted us, please email us at help@keyboard.io if yours is, so we can fix it.

Low quality milling around the thumb keys

In the last update, we talked about how a number of the enclosures the wood supplier sent us for the MP1 run had really, really poor edges around the thumb keys due to ham-fisted hand finishing work at the wood factory.

We’ve had about a half dozen reports of embarrassingly-bad edges around the thumb keys.

Going forward, we’ve made it very clear to the factory that we will reject any keyboard enclosure that has low quality edges.

If the milling around your keyboard’s thumb keys is distractingly bad, please email us at help@keyboard.io to tell us, so we can get you fixed up and give the factory an accurate count of these issues. As with the cracked enclosures, we’d recommend waiting until we have new stock of enclosures before swapping yours out.

A weird LED issue

One customer reported that a block of the LEDs on the right side of his keyboard were misbehaving. We’ve swapped out his keyboard and suspect that this issue was caused by a bad solder joint, though haven’t had spare cycles to dig into this yet.

If your LEDs are misbehaving, please email us at help@keyboard.io.

Other minor issues

We’ve had one or two reports each of scratchy feeling keyswitches, non-clicking clicky keyswitches, missing screws on the stand, beaten up-looking screwdrivers, distractingly bad keycap paint jobs, and incomplete adhesive on the rubber feet for the stands.

We’ve advised the factory of these issues and they’ve said they’ll update their procedures to make sure they don’t happen again.

If your keyboard has any of these issues (or any issue we didn’t mention above), please email us at help@keyboard.io.

Additional keycap sets

The factory confirmed to us this week that the keycap injection molding factory has started work on the additional keycap sets. (This includes both the sets we’ve already promised to you and additional sets that we’ll have for sale later.) After that, they’ll be painted and engraved.

Once they’re done, we’re looking at having the keycap sets sent to you directly from China, rather than running through a fulfillment center in California. We’ve gotten some quotes from local vendors there and, after some initial miscommunication, they’re…a lot more affordable than bringing everything to the US first.

Firmware and Software

The past month has seen a ton of forward progress on the Model 01’s firmware. And there’s a very good reason for that: We’re very pleased to report that algernon is now officially working for us part-time to make Kaleidoscope better.

You can read about what algernon did in his first week here: https://asylum.madhouse-project.org/blog/2017/12/02/kaleidoscope-progress-report/

Generally, we’ve been fixing bugs and improving compatibility. The Numpad mode works much better than it did before, we’ve almost finished up support for 'Boot Protocol’ mode, which will make the keyboard work correctly in your BIOS, as well as letting you type in your FileVault or BitLocker password at boot. We’ve put a bunch of effort into improving the Model 01’s hidden test mode to help the factory catch more issues before keyboard roll off the assembly line.

Simon-Claudius Wystrach, James Cash and algernon have been continuing to push Chrysalis (the GUI) toward release. Jesse’s been working on a small cross-platform “recovery” GUI and flasher for folks who don’t want to mess around with the Arduino IDE or command-line tools.

Documentation

For that past couple weeks, Jennifer Leigh has been writing and coordinating a heroic documentation effort for Kaleidoscope. You can see the fruits of her labor over on the wiki: https://github.com/keyboardio/Kaleidoscope/wiki

Community

https://community.keyboard.io has been active over the past few months. Folks have been sharing everything from thoughts on clever new mounting options for the Model 01 to new firmware plugins and LED effects to a friendly, reasoned discussion of the merits of various key layouts.

We’ll be continuing to post micro-updates to Twitter and as Kickstarter comments as we work toward getting the rest of your keyboards shipped. We’ll be traveling a bit over the holidays, but don’t expect that to have any significant impact on the schedule.

As always, if you have questions, feel free to drop us a note on Kickstarter or email us at team@keyboard.io.