Tcoz Tech Wire

Discoursing on trends and technologies interesting to Tim Consolazio, sole proprietor of Tcoz Tech Services, specializing in Flash/Flex/Air, iPhone, Facebook, Twitter, and related technologies.

"Technology from an indie software developer's perspective".

Thursday, September 17, 2009

Predictable Irrationality: Project Planning by Reverse Engineering a Concrete End Result

This is one of my greatest challenges as a developer. Everybody thinks they know this already. Almost nobody seems to actually practice it. For the most part, this example doesn't use actual technology project examples, but that's deliberate, because oddly, people don't think real life thinking applies to technology project plans.

Anyway, here we go...a client has an idea for a project; they want this fantabulous application that does this, this and this. They qualify you, and hire you. There you are, ready to look over requirements and wireframes, iterate through them, allocate your resources, get some preliminary milestones approved, and create a draft project plan.

The client hands you a few comps; artist renderings of the end result. "That's what I want".

You're a contractor, so of course, you say "erm...have you considered...". Some internal employee that believes he/she has already foreseen the future is annoyed, but concedes, so they rework the comps. You look it over and say, "hmm...but how about errors and alerts...what about some form of tips or help...what will happen if the user does this..." The designer starts to whine, the client gets tired of hearing you say, "have you considered...", and assigns some internal employee to start pushing you to start dev immediately, believing that because they have shown you a snapshot of the end result, that the plan is ready to go.

I want to throw a chair across the room and shout, "EVEN IF YOU SUCCEED, YOU ARE GOING TO FAIL!".

IMHO, it's why most people are accustomed to failure. They don't work towards visions rationally. They get caught up in working around obstacles and situations that prevent them from following the original plan to the concrete end result they want, so they never get there, and abandon the vision; failure ensues. I see people go through this all the time.

Again, you're probably saying, "This is stupid, it's obvious. You have to be flexible, roll with the punches, redirect as necessary". But like a guy I used to work with at American Express used to say, "in practice, common sense ain't so common."

Here's an example; you want to get to California. Somebody sends you a picture of the state and says, create a budget, figure out the wear and tear on the car, and get to work. Oh and btw, it'd be nice to get some pictures along the way.

You say, "well...", the client says, "there's the state, for god's sake you have a picture of it. My internal guy already planned the route, here it is. Get to work."

Well, ok. The route he gave me is N miles, which means Q wear and tear. Ok sir...here's your plan. This is what I'm going to deliver, based on your route. Nice and neat. The client grumbles; it seems more expensive than "his guy" led him to believe, but he accepts the plan. "See? It doesn't have to be so complicated."

Seventy five miles out, there's a detour. It takes you 20 miles out of the way down some other highway. Hey, you're flexible; you redirect as needed...you fight around the detour, take some other roads, and get back to your original plan. You speed up a little to make up the time. Not so bad; a little acceptable risk, but we're ok.

Two hundred miles later, you get a blowout. You're stuck on the road, and have to wait 90 minutes for the AAA guy. You get patched up, but now you're behind schedule. So YOU SPEEEEEEEDDD to make up the time. You may get lucky and make it; you are counting on it. You may get a speeding ticket; you decide to "incur the cost", because you believe it's your fault. You may get into an accident because you are proceeding recklessly. You know it's risky, even stupid; you're risking complete failure, even unrelated personal obligation, but somehow, you feel the whip is justified, because you have to get the plan "back on track".

What about the pictures the client wanted? You take them at 110 mph. The requirement is met, but poorly.

From a project perspective, this is analogous to having your developers work day and night, beyond the point of diminishing returns, because something went wrong. You push them until they just don't care anymore and are ready to quit...or do so. You hire another resource as fast as you can, incur the cost, and crack the whip the second he walks through the door.

All in all, your plan is in the shitter. You are now proceeding by the seat of your pants. Everybody is tense. You've told the client, "just a hiccup, we'll get it back on track." You feel somehow it's your job to "protect" or "insulate" the client from these details. You haven't slept in two days and the scar tissue on the developer's backs makes them numb to the whip. They stop responding, so you push harder.

At this point, getting to the Golden Gate Bridge is a tossup, even unlikely. The budget was blown five states ago. You're way behind schedule for arrival. The client is angry. You're mindset is "too far to turn back now".

If you're lucky, miraculously, you find yourself going over the bridge. It all seems worth it; you shrug and say, "the unexpected occurred, so I failed to deliver the plan, but I did get to CA. I'm hardcore!".

Now the client wants you to come back. Believing he learned what you tried to tell him, but still not really listening, he provides two routes this time, and allocates a little money for repairs and some more time to get back based on how long it took you to get to CA with the originally blown plan. Off you go. Again.

Nothing has really changed.

How would I do it?

"Ok, so, California. When would you like to arrive? Let's propose a route, taking into account it may change drastically based on traffic, detours, etc. Ok, here's my route, and the one you proposed. Here's the good and bad. Either way, if nothing goes wrong, it should take this much time, and this much wear and tear. I can't predict the future though, so let's try to get to this milestone 200 miles away. At that time, I'll call you and report on my progress, and along the way, I'll let you know if I think we're going to encounter any issues, and we'll discuss how to deal with them."

Some scenarios as you try to get to that milestone:

- You make it on, or ahead, of time. No change in plans. Good news.
- You make it, but learn of some condition that will enhance, or complicate, the next milestone. You propose a redirect.
- You make it, but are a little behind schedule. You say that, at this point, unless one of the two above conditions occurs, you will be a day late. See below.
- You make it, or did not make it (accident, etc.), and are so drastically behind schedule that you know the only way to "get back on track" is to be reckless, which incurs risk. You propose abandoning driving and getting to the nearest airport. Regarding the pictures, well, the point was to get to CA. We can figure out the pictures later if you want them; maybe I can get them from Flickr. MAYBE YOU JUST DON'T NEED THEM TO GET TO CA.

Read those scenarios again. If you're a gambler that couldn't afford to lose, would you put your money on good news, or bad?

You're now managing the client's expectations every step of the way; you're communicating effectively. You get a sense of completion at the milestone, you never incurred unnecessary risk or abused resources, and you got the client what he needed; you achieved his vision by abandoning the concrete end result. If he tells you to speed recklessly, you say no. Miraculously, he accepts this. Surprisingly, you work to a new milestone, and the client seems satisfied.

It seems so logical, so easy. But I'm telling you for a fact that almost nobody works this way because people are afraid to tell their clients that they are setting themselves up for failure.

So do yourself, and your client a favor. Be a pro. Don't tell the team to run a pattern you know will probably fail. Work with your client to get them to look at Milestone One as opposed to the concrete end result. Ensure that redirects achieve the core of the vision; recognize what is important, and what isn't, to achieving the vision.

If you want to get to CA, get there safely and rationally. It doesn't matter if you take the bridge...unless the bridge is actually where you want to go. Even then, if you planned to go over it from one direction, but ended up coming the other way, does it matter?

If it does...well, go over it and turn around.

As always, thanks for visiting.

Labels: ,

4 Comments:

At September 17, 2009 8:46 AM , Blogger Marcus said...

As a developer, I have to deal with this same frustration. I don't have a solution, but as terrible as it is, I have some sympathy for the people who get exasperated when we say, "Okay, but what about...?" over and over. I'm not saying that they're right (they're not). I'm saying that I understand what motivates them. If there's a solution, it will have to somehow deal with these motivations -- either re-routing them or quashing them.

When you code, it's generally necessary to plan out a large percentage of what you're going to do before you start -- not the details of the algorithms but all major aspects of the functionality you're coding towards. It's far from trivial to change from one sort of UI to another, so the final UI should be pretty well set before coding begins.

Trouble is, I don't think this sort of thinking is natural to most people. (It is to some of us coders, but we are the freaks. I mean that in a good way. Our "freakiness" -- our anal-retentive nature -- is one of the things that makes us good at our jobs.)

Say you're a cave man and you're about to hunt. There are certain things than you definitely plan out beforehand: what weapons to bring, how much food to carry, etc. But you probably don't map out the majority of what you're going to do. You won't say, "first I'm going to look for a leopard; then, if I can't find one after two hours, I'm going to look for a boar..." You'll just go out and hunt, and you'll deal with new situations as they arise. That FEELS right. It's nice to not have to hold too much in your head at once.

I think that's the natural way (most) people think. They have a general end-goal (bring back food to the cave) and a couple of planned steps (leave cave with spear; walk towards woods), but everything else is improvised.

It's hard to work in a vacuum, by which I mean that most people need SOMETHING TO RESPOND TO in order to make decisions. And that something should be as physical as possible. A wireframe is pretty abstract. It's not something that you can interact with. The natural way to know if you need an error dialogue is to enter an incorrect value in an ACTUAL form and react the fact that the form doesn't respond. Most managers are probably able to work this way with all the members of their team except for the programmers. "Jennings, draft a memo about our first-quarter earnings ... Hm. Having read your draft, I now see that you need to add a paragraph about salary increases at the bottom of page five..."

 
At September 17, 2009 8:47 AM , Blogger Marcus said...

Say I give you a cake recipe. You're reading it over, and I ask, "Do you think we should add more sugar or less?" You'll be tempted to ask, "Can I make the cake the way the recipe says, try it, and get back to you on that?"

(This is a very similar problem to coding, assuming there isn't time or budget to make second cake. Yes, it is MUCH easier to try a first draft and respond to it, but you don't have that luxury. Since you can only make one cake, and since you can't add or remove sugar from a finished cake, your only option is to plan how much sugar to add at the recipe stage. This is necessary but not ideal.)

I deal with pre-planning vs. trying-and-reacting when I direct plays, which is my main activity when I'm not programming. When I'm directing, I'm on the other side of the fence. I try to pre-plan as little as possible.

An actor will say to me, "My scene partner kisses me, and my character is sort of attracted to her but is also married and faithful to his wife. So when I say, 'We can't do this,' do I say it forcefully or sadly?"

I always respond by saying "Choose one option and lets try the scene." Then I react and we do the scene again, often trying the second alternative. This is a MUCH more natural way to work than to pre-plan many steps ahead.

I believe we would code this way if we could. If I could whip up a complex app in an hour, I wouldn't ask all the "but what about...?" questions. I would just code exactly what the wireframe suggested, run it up the flagpole and see what happens. Even if it turned out to be total crap, it wouldn't be a major hassle to start over from scratch and have a second draft done in another hour.

Alas, coding doesn't work like that, so we are forced to think in an un-natural way -- a way that is more about pre-planning than trying and reacting.

There are other human activities that demand this approach. Architecture is one. You can't just jump in and build a foundation without having first planned out the entire building. This is why architects are rare birds. Not everyone is cut out to be one, because not everyone thinks in this way.

I'm one of the rare birds who does, and I find that it affects even the way I converse with people.

 
At September 17, 2009 8:47 AM , Blogger Marcus said...

Often, people will ask me to do something (not necessarily a programming task), and they'll ask in a vague way -- not fully explaining what they want. For instance, they'll say, "Marcus, would you go get that thing and put it in the closet?"

What's fascinating to me is to watch how other people tend to deal with requests like this. Over the years, I've noticed that they often don't ask "What thing?" and "Which closet do you mean? We have three."

They just form a hypothesis (maybe they do this unconsciously) about what the thing is go get it and they randomly put it in one of the three closets.

The person who make the request later finds the wrong thing in the wrong closet and says, "Sorry, I didn't mean 'put the suitcase in the hall closet', I meant 'put the shoe box in the bedroom closet.'"

What amazes me -- given my personality type -- is that neither party is upset. They both seem comfortable with a form of communication in which they grope forward from a vague concept to a specific one.

Whereas I'm much more comfortable with specifics up front. I don't want to waste my time trying to figure out what the thing is and what closet to put it in. So I say, "Sorry, can you tell me what thing you mean and which closet you want me to put it in?" I'm often surprised when my request for specificity is met with irritation.

But though I'm right to request specificity from the point of view of minimizing confusion and needless work, I'm wrong from the point of view of standard thinking and conversation. My eccentric way of thinking is one of the things that makes me a good coder.

Getting back to the problem, regardless of what's natural, we CAN'T improvise code. There isn't time. We MUST plan out most of the app before coding begins. That may be unnatural for many people, but it just is the way it is.

Like I said at the start, I don't have a solution. But I have found some approaches that minimize the problem.

People who don't think like me hate being boxed in. They hate feeling like they have to come up with a finished plan NOW and that they can't change anything later. I've found that this is generally more of a feeling than an actual business problem. Even if it's unlikely that anything WILL need to change, people still want to feel that they have the option of changing.

So I usually say this, "Once I'm done coding, there will be some things that will be really easy to change if you don't like them. It will take me five minutes to change the background color or the font or the blah blah blah ... but it will be very expensive to change the dropdown menu to a series of check boxes, so we kind of have to figure that out now..."

I get much better results if I dole out some flexibility at the same time as I demand specificity.

We coders also need to keep in mind that non-coders have no idea what's hard to change and what's easy. Why is changing a background trivial while changing a dropdown to a bunch of checkboxes complex? It SEEMS like it should be easy.

We can't make it easy, but we can warn the people we're working for about the fact that it's not easy. When I do this, I usually acknowledge the fact that this reality is less-than ideal: "UNFORTUNATELY, it's going to be really hard to change X later, so are we sure that's the way we want to go?"

For programmers and managers to work well together, managers need to understand that programming requires a unique mindset and some un-common planning requirements; meanwhile, programmers need to understand that they are being hired because they are specially gifted at thinking in a non-standard way. They need to learn how to help their co-workers see why things need to be done in a way that is unnatural to them.

 
At September 17, 2009 8:48 AM , Blogger Marcus said...

Often, people will ask me to do something (not necessarily a programming task), and they'll ask in a vague way -- not fully explaining what they want. For instance, they'll say, "Marcus, would you go get that thing and put it in the closet?"

What's fascinating to me is to watch how other people tend to deal with requests like this. Over the years, I've noticed that they often don't ask "What thing?" and "Which closet do you mean? We have three."

They just form a hypothesis (maybe they do this unconsciously) about what the thing is go get it and they randomly put it in one of the three closets.

The person who make the request later finds the wrong thing in the wrong closet and says, "Sorry, I didn't mean 'put the suitcase in the hall closet', I meant 'put the shoe box in the bedroom closet.'"

What amazes me -- given my personality type -- is that neither party is upset. They both seem comfortable with a form of communication in which they grope forward from a vague concept to a specific one.

Whereas I'm much more comfortable with specifics up front. I don't want to waste my time trying to figure out what the thing is and what closet to put it in. So I say, "Sorry, can you tell me what thing you mean and which closet you want me to put it in?" I'm often surprised when my request for specificity is met with irritation.

But though I'm right to request specificity from the point of view of minimizing confusion and needless work, I'm wrong from the point of view of standard thinking and conversation. My eccentric way of thinking is one of the things that makes me a good coder.

Getting back to the problem, regardless of what's natural, we CAN'T improvise code. There isn't time. We MUST plan out most of the app before coding begins. That may be unnatural for many people, but it just is the way it is.

Like I said at the start, I don't have a solution. But I have found some approaches that minimize the problem.

People who don't think like me hate being boxed in. They hate feeling like they have to come up with a finished plan NOW and that they can't change anything later. I've found that this is generally more of a feeling than an actual business problem. Even if it's unlikely that anything WILL need to change, people still want to feel that they have the option of changing.

So I usually say this, "Once I'm done coding, there will be some things that will be really easy to change if you don't like them. It will take me five minutes to change the background color or the font or the blah blah blah ... but it will be very expensive to change the dropdown menu to a series of check boxes, so we kind of have to figure that out now..."

I get much better results if I dole out some flexibility at the same time as I demand specificity.

We coders also need to keep in mind that non-coders have no idea what's hard to change and what's easy. Why is changing a background trivial while changing a dropdown to a bunch of checkboxes complex? It SEEMS like it should be easy.

We can't make it easy, but we can warn the people we're working for about the fact that it's not easy. When I do this, I usually acknowledge the fact that this reality is less-than ideal: "UNFORTUNATELY, it's going to be really hard to change X later, so are we sure that's the way we want to go?"

For programmers and managers to work well together, managers need to understand that programming requires a unique mindset and some un-common planning requirements; meanwhile, programmers need to understand that they are being hired because they are specially gifted at thinking in a non-standard way. They need to learn how to help their co-workers see why things need to be done in a way that is unnatural to them.

 

Post a Comment

<< Home