Why do we need goal setting and planning? In general, how can it be used effectively? The answer to the former is that we need to know that we're heading in the right direction as fast as possible. This need has three components:
Goals are traditionally used as the solution for all three; however, they work best for direction. We can make all our lives a lot better if we focus our goal setting on fulfilling the need for direction while using other methods that are better suited for magnitude and validation.
Specifically, magnitude is best achieved by creating a culture that fosters intrinsic motivation. Then people will maximize their magnitude without overhead supervision. In addition, effective planning ensures that people work together efficiently.
Validation actually is a smaller need than people think, and it's best served by a combination of 1) not worrying about it so much, and 2) having the trust to rely on qualitative discernment instead of hard metrics.
Note: This essay is written in the context of working on a software development team; though it should apply in the general case also.
To use goals effectively, first recognize that the reason you're using them is to help you prioritize—not to provide motivation or accountability. Thus, measurable and time-bound goals aren't necessarily that important. Those kinds of goals can be helpful in specific situations but shouldn't be required in all situations. Once you stop using goals badly, you're half-way to using them well.
That is the most important thing. With a clear purpose in mind for goals, how to actually use them will come naturally as you try different things and continually evaluate their helpfulness. For myself, I currently use three tiers of goals: long-term (i.e. life-long), mid-term (on the range of a few months to a year) and short-term (weekly) goals. Because I don't focus on measurability or time-boundedness, you might think of my short-term "goals" more as "tasks." For the most part, short-term goals lead into mid-term goals, and mid-term goals lead into long-term goals.
For example, here are my goals as of this week:
After making my short-term goals, I then schedule: I assign the tasks to specific days and think about the details (e.g. the Chinese visa application has several parts I need to take care of). The specifics of the framework are up for debate, but it's lightweight and it helps me prioritize effectively. The goals are actually helpful and I refer back to them often.
On a software development team, mid-term goals might focus on projects we're trying to complete during the quarter. If the team practices scrum, short-term goals are just stories on the current sprint.
As a last note, remember that even the short-term goals still aren't time-bound by default, even though they are set on a regular basis. The list of short-term goals is intended to be the complete space of things we need to think about. As you adjust your plans from day to day, you just look at the list and figure out which things should be done first. If you don't finish the list by the end of the week, that's OK. On a Scrum team then, the stories you take on in sprint planning should be everything that you hope to accomplish during the sprint, not a hard commitment on what will actually get done. The need to make sure you're actually getting things done is handled by a separate validation system.
For further reading, consider The 7 Habits of Highly Effective People (specifically, the first three habits), The Power of Full Engagement and Essentialism.
Once you know what to work on, how do you get it all done? There are two sides to this equation:
Maximizing work done in turn has two parts: quantity (time spent working) and quality (the effectiveness of that time). Quantity typically isn't an issue. Quality is largely affected by motivation. The book Primed to Perform gives an excellent presentation of the large amount of research that has been done on this topic. As the authors say, "Why we work affects how well we work." The short version is that there are six common sources of motivation, three of them good, three of them bad. You must minimize the bad motivations while maximizing the good.
The good motivations are:
The bad ones are:
To understand how these motivations affect performance, you need to understand two different kinds of performance. Primed to Perform calls them tactical and adaptive performance. Tactical performance is related to efficiency; it's how well you can execute a plan. Adaptive performance is how well you can diverge from a plan in order to handle "VUCA" (volatility, uncertainty, complexity and ambiguity). The more VUCA is involved in an area, the more adaptive performance is needed. All the motivators can drive tactical performance, but the bad motivators hurt adaptive performance while the good ones increase it. Example: giving someone a raise for a job well done (economic pressure) might help them work faster, but it won't necessarily help them adapt to changing circumstances.
To maximize the amount of work done, we need to consciously think and talk about these motivators. Periodically review the influence these motivators have on yourself and on your team or organization, and figure out how to improve things.
When working within a goal-setting-and-planning framework like OKRs or Scrum, be aware of how the framework impacts these motivators. A danger of using goals for motivation is that it can undermine the good motivations while strengthening emotional and economic pressure. You shouldn't feel bogged down or restricted by the framework. It should be your servant, not your master.
And you really should just read Primed to Perform.
The second part of magnitude, minimizing the amount of work needed, is where planning comes in. Part of this is simplicity in design so that don't set out to accomplish some hard, complex, unnecessary solution. The other part is figuring out how to get N people to work efficiently together. I have some thoughts on the latter with regard to software development, but I'll spare you the details. The important thing is to recognize that this is a problem and you need to think hard about how to do it best.
Stephen Covey said it best in The 7 Habits:
It is much more ennobling to the human spirit to let people judge themselves than to judge them. And in a high-trust culture, it's much more accurate. In many cases people know in their hearts how things are going much better than the records show. Discernment is often far more accurate than either observation or measurement.
The most important thing to remember about validation is to not let it spoil your goal setting and planning. Think about how terrible university education is. Learning is one of the most compelling of all activities, shouldn't the opportunity to work full-time on just learning be a wonderful thing? The problem is that formal education tries not only to teach you something but to verify and measure your learning. General education is hit the hardest. How great it would be to spend all day reading books that contain a distillation of the world's best thinking. Instead you have to read a textbook filled with random facts and prepare for a multiple-choice test.
So don't let that happen to your organization. If you happen to be working on something that's measurable, then sure, measure your performance. Sales, losing weight, and your golf game are all great candidates for measurable goals. But don't try to force a square peg into a round hole by coming up with strange metrics for things that are better discerned.
I handle validation for myself by giving a weekly color rating—green, yellow or red—to each of my mid-term goals. I'll often give a preliminary rating once or twice during the week, but I'll give the final rating at the end of each week. When I have non-green ratings, I then think about if there's a problem with direction or magnitude (or neither). I might decide to give higher priority to a goal with a red rating the following week; i.e. I modify direction. Or I might try to identify a performance barrier that I can fix in order to increase my magnitude. Or, if my highest priority goals are green and the reds are occurring elsewhere, I might accept it as not a problem. If I consistently get reds on a low-priority goal, I might simply remove the goal.
During validation, you should also think about the urgency of your goals. Most of the time, we want to stay in quadrant II of the time-management matrix described in The 7 Habits. Re-prioritizing goals and eliminating performance barriers are both quadrant II activities. However, quadrant I activities (like working overtime) are needed when urgent and important goals are slipping. You need to decide if a goal that's slipping is becoming truly urgent and take action if necessary. After the goal is complete, you should reflect on what you could have done to prevent the goal from becoming urgent/what you can do in the future to prevent similar situations.
And that's really all there is to it. You could easily adapt this to a team by having everyone vote periodically on each mid-term goal and then talk about it. (You could even convert the colors to numbers in order to satisfy people who like metrics). Or you could come up with another system. The main thing is to remember the principles:
From here to there
A great way to start out could be setting a mid-term goal to have great direction, magnitude and validation systems (and in your planning meetings, you can rate each system from red to green!). In summary, what questions might you ask yourself whilst evaluating your direction, magnitude and validation systems? Perhaps these:
Separating these needs out and talking about it periodically will help a lot in the conversation of how we should be using goals and plans.