Programming Blog

Jeremy Morgan

Mostly Coherent Ramblings of a Silicon Forest Software Developer

Transforming Your Organization With the Andon Cord

Imagine you’re working in a factory. You’re assembling Toyotas all day long, then your part won’t fit. What’s going on? You do this hundreds of times a day but now the bolts won’t go in. No reason to panic, you pull a cord to get help. Two co-workers arrive immediately. They find out you have a box of bolts with the wrong thread. They swap out the bolts, and you keep going.

"How the Andon Cord can Transform Your Organization"

At the next meeting, you discuss what happened with the team. The two boxes of bolts are the same color with a similar-looking label. It’s easy to mix these up, and that’s exactly what happened. The team puts different colored labels on the boxes so the folks stocking your bolts don’t mix them up.

Sounds like a great system right? You ran into an issue, called for help, and with more eyes on the problem you found a solution. Then you share the solution with the group.

That’s the power of the Andon cord, and it’s something you can use in your IT organization.


The Origins of the Andon Cord

The “Andon Cord” is a part of the Toyota Production System. It was created when Toyota was smashing records for production, and they did something unheard of. It was not intuitive or natural, a way to stop the production line when it was common knowledge you.. don’t stop the production line.

"How the Andon Cord can Transform Your Organization"

It was just a simple rope, like the one you pull on a bus to let the driver know your stop is coming up. People pulled it when they needed help. The ways it changed their manufacturing efficiency was astounding.

Though it sounds simple, this was a big culture change for the factory. Anyone on the floor had the power to pull the rope for any reason. They weren’t scolded or chastised for doing it. You wouldn’t get fired for pulling it too much. You just pull.

The results of this experiment? They created a learning organization.

In the words of Andrew Shafer: you’re either a learning organization or you’re losing to someone that is.


My Personal Experience with the Andon Cord

I’ve been blessed to see the entire spectrum of this philosophy. Right out of high school I worked at a lumber mill. My job was to pull boards off a moving chain and stack them. 10 footers in one pile, 14 footers in another, etc. We had big red buttons you would press to stop the chain when things went wrong. Usually, it was a row of lumber falling over, or being unable to stack the incoming boards fast enough.


"How the Andon Cord can Transform Your Organization"



If you stopped the chain, instantly things went quiet, and beacon lights all around you went off letting everyone know it’s stopped. If we stopped the chain for more than a minute, the planer and saws would have to shut down, and our former drill sergeant boss would come down the stairs screaming, asking why the chain stopped. They could fire you for doing it too many times. Though I wasn’t fired for it I remember being on the receiving end of some serious yelling for stopping the chain. I can still picture him screaming and spitting in my face yelling.


"How the Andon Cord can Transform Your Organization"



As a volunteer firefighter for our local fire district, it’s quite the opposite. If you run into a problem or large abnormality, you must pull that Andon cord and ask for help. If you come on to a scene and don’t know what to do, or have a patient in a bad situation and you need ideas, you had better squeeze that radio microphone and ask for help. People’s lives are at stake, so it’s not just encouraged it’s a mandate. It is an expectation that you stop what you’re doing and ask for help.

After the call an officer or chief will do a “blameless post mortem” and discuss why you asked for help, what you could have done, and whether the course of action was successful. We all learn from it and become better firefighters/EMTs. Learn, learn, and learn some more.

In my professional IT career I’ve seen a variety as well, but these two are the extremes. You’ll need to find your sweet spot for your organization.


Can You Use This?

So implementing this in your organization in principle is easy, at least in principle. When someone runs into a blocker, they alert the team they need help. You could do this via Slack, IM, or even email if you absolutely had to. Then the team drops what they’re doing and everyone swarms the problem until it’s solved.

You might be rolling your eyes already. You will find resistance to this idea, and the reasons are pretty easy to guess.

The Coworkers or “doers” might object because:

  • Nobody wants to just drop what they’re doing for your problem.
  • They’ll be too busy, or in a meeting.
  • They don’t believe swarming will solve the problem.

Managers might object because:

  • They don’t want a bunch of doers stopping work at once
  • They may not believe swarming is effective
  • Timelines may be affected by this.

These are all valid concerns. Honestly, these may be valid reasons for your organization to avoid doing it. You should at least give it a try.

The upside here is you become a learning organization, constantly experimenting, learning, and advancing your effectiveness. It’s highly unlikely you’ll pull the Andon cord and stop work for something you’ve already solved as a team, right? It’s worth trying.


How You Can Implement the Andon Cord

If you want to implement this idea with a team, here are some things you’ll want to establish before starting out.

  1. Identify your fixers - Identify common pathways for common problems. For instance if it’s a database issue, get Mark involved. He’s great with databases. If it’s an infrastructure issue, get Debbie involved. She knows our systems like nobody else. Identify your “clutch” people, chances are your team already knows who they are, but establish it formally.

  2. Prioritize your problems - Categorize the severity of the issue and determine a level that requires stopping everything. If it’s slow performance or a small configuration issue, you can put it under the threshold for stopping the team. If it’s a major blocker that prevents progress, bump it up in priority so everyone knows they’re stopping for a good reason.

  3. Store your findings immediately - The first time you “pull the Andon cord” document your findings and the solution right away. Don’t wait for it, and show the team the results. This helps you greatly with buy-in of the idea when they can see the usefulness.

Conclusion

Nobody has to tell you that technology moves fast. If you and your team want to keep up you need an advantage. By creating a culture that fosters experimentation and constant learning you’ll be leaps ahead in the battle to keep up. The days of “we’ve always done it this way” are over. This is one tool you can use to make your team more solid an effective. Try it out and let me know what you think. You can always yell at me on Twitter or LinkedIn and start a thread. I’d love to hear your experiences and feedback.

Comparing and Syncing IIS Configurations

Imagine you’re an administrator at ACME Widgets and it’s time to upgrade your IIS server. You’ll just copy over some folders and point the DNS to the new server and be done right? If you’ve ever done this before you know that isn’t the case. The new IIS server needs to be configured identically to the old one or you’re going to have problems, and you don’t have time for problems.


Using DISM to Create a Repeatable IIS Installation

Repeatable installs are all the rage in Devops these days. As developers we have this “automate everything” mentality, and for good reason. In this article I’ll show you how you can do that with IIS installation as well. There’s no reason to go hunting and pecking around the GUI every time you need to do this. This is the just one of many ways to automate IIS installs, which I’ll be covering in the next few weeks.

I also cover this in depth in my latest Pluralsight course on Installing IIS.

Unit Testing With Dotnet Core

So you’ve just started building .Net Core applications and really starting to gain some traction. You quickly learn how mature and thorough the .Net Core framework is becoming and think “I need to start writing some unit tests for this!”. As it turns out, it’s super easy and very intuitive, especially for C# developers.

Set Up Easy File Sharing With Samba

In this tutorial, I’ll show you how easy it is to setup file sharing on your network using SAMBA. You can easily share files between Linux and Windows machines with a pretty minimal amount of setup.

Windows to Go With the Spyrus USB Drive

What we need is a good bootable live USB stick for Windows. I’ve said this many times over the years, and hacked together things to make that very thing happen, but nothing that worked really well. So when the folks at Spyrus sent me a Windows to Go USB to check out, I was pretty excited.

Which Distribution of Linux Should I Use?

I’m often asked this question: “hey, you’re a Linux guy right? What Linux should I use? I have this friend who recommends _____ and I want to know what you think?” I usually reply with the same question: what do you want to do? So I decided to make a blog post about it that I can send people instead.

My Latest Virtualization Setup

Like many geeks of the time I spent the 90s and 2000s with at least 2 or 3 old computers in a closet, connected by a switch running various operating systems with various services running on them. Giant, loud, clunky machines whirring away.

Load Testing Your IIS Web Server

All the theory, calculations, and estimations in the world aren’t going to tell you how your website will truly perform under a load. If you’re deploying a new server, or doing any kind of performance enhancements you don’t want to test your results in production. It’s always a good idea to see how your system behaves before your visitors do. To do that, you can use a load testing tool, and here are a few I use quite frequently.

Update: I’ve featured these tools is my latest IIS course on Pluralsight, IIS Administration in Depth, check it out!

How to Install Microsoft SQL Server on Ubuntu Linux in 5 Minutes

I must admit I was surprised when I learned that Microsoft SQL Server would be availble in Linux. They’ve been pushing the open source initiative hard, but I didn’t expect something this big. Oh yeah, Visual Studio is now available for Mac as well. I just saw a pig flying by.

While MS-SQL is not open source they have made it available to run on open source platforms such as Linux and OSX, which I can imagine took a ton of work. So I decided to take advantage of this new option and try it out. It works great! It took 5 minutes to install. Here’s how you can do it too. Note that you will need a server with 3.5 gigs of RAM for this.