HPT-Assessment


resolver

This article is a great example of the need to include all levels of employees.  It shows that you get better results when include frontliners in the process of identifying and resolving performance issues.  I want to emphasize the role of the supervisor in this process.  Notice the process begins when “a supervisor facilitates the team to define a performance issue that is under the team’s control.”

Its not the Training department or the performance specialist who initiates the effort.  Its the supervisor.  This is evidence of an organization that is committed to performance improvement. The days of centralized training and performance improvement are long gone.  Markets and workplaces move too quickly to rely on a separate department to identify and address performance issues.

In other words, performance improvement is a mindset.  It’s either in an organization’s DNA or it isn’t. I’ve been doing this long enough to know that all the collaboration and development from the Training department (which is where the performance improvement specialist is often assigned) cannot change an organization’s culture all alone.  Supervisors and their upline have to make it a priority and look for opportunities to improve.

How do you instill this mindset?  First, performance improvement has to be a part of the everyday conversation.  Below are some questions you can ask regularly to get the conversation started.

  • What can we do better?
  • What are we missing?
  • Who else can we ask about this?
  • Where are your pain points?

The second thing to do is let down your guard.  Humility and a willingness to accept feedback takes practice.  Supervisors have to demonstrate this on a regular basis.  The best feedback hurts a little.  Sometimes it hurts a lot.  A little short-term pain is worth it if you really want to perform at a higher level.

Third, learn.  There is a direct link between learning and growth.  When a person stops learning or loses the desire to learn personal growth stagnates.  The same thing happens to an organization.  There is no shortage of things to learn in an organization.  There is a risk that you might be focused on the wrong thing to learn about.  It won’t take long to find out what is worth learning about and what isn’t.

Finally, maximize the resources you have.  Expertise is everywhere.  You just have to look for it.  Despite their day-to-day responsibilities, most people have skills that are un-utilized or under-utilized.  Tap those resources.  Many times you will find skills and perspectives you never would have known about.  A hidden benefit to this is increasing the satisfaction of the employee.  We all toil away at our job doing work we have to do to keep the operation running.  Giving someone a break to do something different feels like a reward.  For a few hours or days they get to do something they really love and are good at doing.  At the end, make sure to recognize their efforts.  That will motivate others and motivate others to share ideas to improve.

As you can see from the description above, performance improvement can become self-sustaining.  Employees have to feel that the people they report to are interested in their perspective and willing to act.  Once this gathers momentum the culture of an organization will change.  This sounds scary but it doesn’t have to be.  The change I am talking about is the atmosphere in the workplace.  Organization-wide performance improvement is empowering.  It puts a spring in people’s step.  It breeds confidence. It make people happy at home and at work.  Who doesn’t want that?

I just read a well-written article on front-end analysis.  In addition to providing practical insights into performance analysis the author does a good job differentiating human performance technology (HPT) from basic training development.

Human performance technology is a set of disciplines but it is also a way of thinking.  A thorough performance improvement effort requires specialized experience but everybody in an organization can help.  One of the best contributions a person can make is to conduct an informal front-end analysis.  Note the emphasis on informal.  This may sound intimidating or complicated but it isn’t and doesn’t have to be.

The author provides two goals for the front-end analysis but anyone can make significant progress on the first, defining the current and desired performance.  Most performance improvement efforts start with a problem.  The person who identifies the problem can probably provide valuable information about it and its causes without the guidance or involvement of an HPT professional.

Below are relevant questions I chose from the article.

  • Based on what evidence can you say you have a problem?
  • How will we know when the problem is solved?
  • What are the possible causes of the problem? (Lack of data, tools, incentives, knowledge, capacity, motives)
  • What is the probable cause? (Of all the possible causes which one is the most likely?)
  • Should we allocate resources to solve it?

The point of this process is to make an early attempt at understanding a problem.  In most cases these questions will bring focus and help determine if more people need to get involved.

ISPI publishes a weekly digest of news and information for performance improvement consultants.  Last week they published their top stories for 2011.  Over the next few days I will feature my thoughts on those I feel are particularly noteworthy.
Click here for my first entry.

One of the ironies of being an expert is that you often lose touch with what it is like to be a novice.

In October Harvard Business Review posted an article titled “The Best Approach to Training.”  With a title like that its bound to get a lot of hits.   While the article has value, its title is a little misleading.  Rather than focus on the latter I will focus on its virtues.
NOTE: I quote liberally from this article.  I am not trying to take credit for the author’s work.  Sometimes it is difficult to maintain a flow and quote the author fully.  This is more obvious in some places that others.

The setup: “Experts often are unable to articulate the many “obvious” (to them) things they do when carrying out a procedure or solving a problem.”  When tasked with training a new employee an expert tends to gloss over important information or steps because it has become second nature to them.  This actually makes the “trainee” more dependent on the expert/trainer, the exact opposite of what they set out to do.   As a result, the organization is not getting maximum productivity of either employee.

The diagnosis: According to the author, the common approach is to adopt an academically developed and tested technique to improve training and instruction that likely involves a resource-intensive task analysis.  This makes sense if you are documenting the minimum qualifications for a job, establishing performance criteria, and creating a training program. However, if you are simply training a new employee I believe you can accomplish more by getting him or her into the flow of the organization first.  After a short period of assimilation you can shift focus to mastering the individual tasks of their job.

The solution: “The best way to identify what experts do is to have them solve the problems or carry out the tasks in question and to require the expert to justify the steps he is taking as he takes them.”  This approach takes a macro-level view and enables the expert to consider the work holistically.  As a result, the resulting training bears a greater resemblance to the reality of day-to-day responsibilities.

An example:  The author asked experienced college physics instructors to create a set of problems — representing a part of the course — that a student in introductory physics should be able to solve if he or she “understood” that part of the course.

He then asked them to solve the problems and narrate their steps.  While they were talking he was “furiously” capturing their thoughts. In his words, “the instructors [often] had to stop and scratch their heads as they tried to provide a justification for their steps. The justifications in this case were rooted in laws of physics, but the relevant features or implications of the laws were things that the instructors had internalized or automated and they struggled to make them explicit to me.

The resulting solutions were of course quite lengthy and verbose, but they ultimately provide the raw material for guiding the construction of better worked examples and lecture materials for learners. I use the notes to then solve new problems with the expert available to help me when I don’t know what to do. Each time I reach an “impasse” I revise the notes. Ultimately, I reach a point where I can solve all problems the expert gives me.”

My thoughts: I have used this method and found it to be useful.  In fact, my most successful projects occur when I can think like the expert.  No one will confuse me for being an expert, but going through this process enables me to apply my skills to the subject matter and create relevant learning opportunities.  The challenge is coming up with the problems in a work setting.  In the end I don’t think this is approach is less cumbersome or resource intensive but the outcome is better.

One of the first questions stakeholders ask in the learning development process is, “how much training credit are we going to give for this?”  This is actually one of the last questions I ask when developing a training solution.

Non-developers ask this question first because they are usually accountable for creating training opportunities that will help employees meet their annual requirements or because they have to fit the training into a larger program.

Developers ask this question last because it places the emphasis in the wrong place, limits the design process, inhibits creative thinking (yes training development requires creativity), and because it is impossible to answer at the beginning of the process.

In Chapter 9 of Designing Successful E-Learning, Michael Allen describes a process he calls “backgrounding.”  This is an information gathering stage that occurs before the actual design of a training solution begins.  In this process he suggests some key questions you SHOULD be asking first regardless of the format of your training solution.

  1. Is there really a human performance problem? This is similar to standard 5 of ISPI’s performance standards which I wrote about here.
  2. Is the problem caused by a lack of ability to perform?  Cause analysis is standard 6, which I wrote about here.
  3. What are the determinants of behavior?  “People do things for reasons.  Their reasons may be built on misconceptions, fears, and lack of confidence, desire to fit in and behave like everyone else, perceived strengths and so on.  To be successful with a learning intervention, instructional designers need to identify the primary determinants of the behaviors that need to be changed.”

When I am “backgrounding” a project I want to know what result we are trying to achieve, what problem we are trying to solve or what opportunity are we trying to address.  ISPI uses the acronym RSVP to describe this process.

R – Results
S – Situation or context
V – Value
P – Partnerships

The answers you get when backgrounding a project will focus your efforts and greatly increase your likelihood of success.  Forcing a topic (assuming it is the right topic) into a predefined time-limited box will greatly impair your design and development effort and may cause you to miss your target altogether.

A key word in this series is “design.”  It would be understandable for a person to ask “when is he going to get around to discussing design?”  I’m almost there.  Before I do, I want to review where we have been in previous posts.

Whenever an organization is dealing with a performance problem, the focus must be on results (ISPI standard #1).    You may think training is required.  You may discuss who is or is not doing what they’re supposed to do.  Some may suggest investing in new software or systems.  Resist these temptations.   Focusing your efforts on results will put your discussions into the right context.  This will enable you to collect the right information, understand the true cause of the problem, and come up with a solution that will achieve the desired results.

Focusing on improved results sets the tone for the entire effort.  You must also consider the situation or context (ISPI standard #2) and decide what resources are required to effectively achieve the desired results.  With a clear understanding of the context and the right people on the project (ISPI standard #4), it is time to do a detailed analysis of the problem (ISPI standard #5).  Your preliminary research and partnerships will help.  Throughout all of this, resist the temptation to draw conclusions too soon.  Patterns will emerge.  Solutions will seem appropriate and attractive.  Wait until you have all the data and have analyzed it before you draw conclusions.  Let the data reveal the true nature of the problem and what is causing it (ISPI standard #6).

I can’t tell you the number of times I have seen an organization decide training will solve a problem without any idea what results they are trying to achieve.  They see that something is going wrong so they automatically assume training will fix it.  What do you train on?  Who decides what the training should be about?  How will you know the participants got what they’re supposed to get?  To find the right answer you have to ask the right questions.  Here are some ideas to get started.

  • What are your expectations (be specific)?
  • What does “wrong” look like?
  • What does “right” look like?
  • Who is doing it “right?”
  • Why is this person doing it “right?”
  • Why can they do it “right” and others can’t?
  • How will we know we have achieved our goal?

This process does not have to take months to complete.  Depending on the scope of the situation, performance improvement can be achieved in weeks or possibly even days.  Do not automatically assume that this process will consume a lot of time and resources.  It is not unusual for this process to save money.

This is a time-tested approach to achieve a successful outcome.  It isn’t always glamorous, but it works.  With the review behind us, lets get on with the discussion of ISPI standard #7, design.

…one common occurrence that warns you that a performance discrepancy may be lurking around is the announcement that takes some form of “We’ve got a training problem.”  Someone has detected a difference between what is desired and what is actually happening.

But statements such as “We’ve got to train/teach…” are pits into which one can pour great amounts of energy and money unproductively.  Such statements talk about solutions, not  problems.  Training (teaching, instruction) is a solution, a remedy – a procedure used to achieve desired results.  It implies transferring information to change someone’s state of knowledge or ability to perform.

But lack of information is often not the problem.

Robert Mager/Peter Pipe, Analyzing Performance Problems, p. 8

If lack of information is not the problem how do you find out what the problem is?  One of the first questions one should ask when looking at a performance problem is, “what does ‘right’ look like?”  In performance terms “right” is defined as optimal performance.  Actual performance is the current way the work is being done.  When starting a project, one should not assume the problem lies is the “actual” way the work is being done.  One should also not assume that training will resolve any perceived problem. The problem may have a cause nobody has considered that cannot be resolved through training alone.

A fundamental step in understanding a performance problem is documenting the actual and optimal performance.  The difference between actual and optimal performance is the performance gap.  In many organizations it is likely that optimal performance has been documented at some point.  This can be in the form of a manual or an existing training course.  One should consider that the problem may be caused by a lack of accountability to existing standards or failure to send people through existing training.  Reviewing existing documentation can provide a good starting point for any needs analysis.  It provides valuable background and a reference point from which to begin data collection.  However, one should also consider other factors that may be causing a performance problem.

The focus of this post is how to document and analyze a performance gap.  I will cover how to close a gap in a later post.

There are many ways to approach a gap analysis.  To effectively understand an issue a combination of analysis techniques is usually required.  Below are seven types of analysis ISPI recognizes:

  • Job or Task Analysis—Identifies the required activities, information, processes used, and outputs produced and then compares that to actual practice.
  • Process Analysis—Identifies the cycle time compared to process time; time at task compared to time on rework, waiting, or checking; resources consumed and the cost of those resources; and what drives activity (customer or product requirements).
  • Work Environment Analysis—Identifies and evaluates the effectiveness and efficiency of feedback, the reward and incentive system, information and communication systems, work and process designs, and work tools and equipment.
  • User or Audience Analysis—Identifies current expectations, perceptions, physical capability and capacity, and knowledge and skills.
  • Communication Systems Analysis—Identifies and evaluates the availability, capability, capacity, upgrade ability, and cost to use and maintain.
  • Market Analysis—Identifies the size, competition, growth, current and potential constraints or limitations, organizational expectations, initiatives, capabilities, and capacity.
  • Data System Analysis—Identifies and evaluates the capability, capacity, availability, upgrade ability, and cost to use and maintain.

Speaking from experience it is unlikely that a performance specialist would possess all the skills required to successfully complete each of these forms of analysis.  This is one reason why ISPI emphasizes partnerships.  If the stakeholders on a project feel strongly that any of the methods listed above are necessary to gather data they should also be prepared to employ a specialist to complete the data collection.  Although the work can be done by a layperson it will add time and may impact the overall quality of the data.  Ultimately the decision is a matter of cost and the benefits derived from the expenditure.

Regardless of who is conducting the analysis, the participants play a critical role in the quality of the data.  Participants should be representative of the entire team in knowledge, experience, and responsibilities.  A control group can be helpful in validating and clarifying data.

Access is another critical success factor.  Being able to observe and interact with job performers in their work environment may be necessary.  However, striking the right balance between observing and interacting can be a challenge.  In my experience it is very important to be open with the participants about your needs and expectations.  It can be unsettling for an employee to be observed.  This is where it is critical to have a sponsor who can communicate with employees and address their concerns.  Interacting with participants may influence your data so should be considered with caution.  Direct contact with participants can also negatively impact their productivity which may also negatively impact the data.

Once optimal performance is documented, the analysis should attempt to validate while documenting the actual performance of workers.  This is where the control group is useful.  It is possible that changes have taken place since previous standards were written that require an organization to revisit their expectations (optimal performance).  Finding out how to compensate for the gap is where the needs analysis can get complicated.  It would be a mistake to assume the gap is caused by a lack of knowledge.  Unfortunately, many organizations treat all performance issues this way.  If the problem is caused by performers lacking knowledge, it is logical to assume the solution is traditional training.  However, there may be another cause.  But that is a discussion for a future post.

I am continuing in my review of ISPI’s performance standards. The remaining standards follow a systematic process that is familiar to instructional designers, who refer to it as ADDIE (Add-ee).  This process consists of five steps or phases: Analysis, Design, Development, Implementation, and Evaluation.  This process has stood the test of time and remains the basis for most instructional design efforts or discussions.  ISPI differ slightly with the addition of a standard for requirements.

Needs assessment or analysis is ISPI’s fifth performance improvement standard (the A in ADDIE).  This is a vast topic.  Entire books are written on this subject.  Although it would be impossible to provide a thorough description of needs analysis in this space, I will attempt provide enough information to help you understand the factors to consider and the procedures to follow in a successful analysis effort.

In my experience, the needs assessment begins with standard two, context.  In my post on that standard I wrote, “identifying and discussing potential barriers will help design an intervention that will achieve the desired outcome.” Barriers to performance are often indicators of what is causing a performance problem.  Although one must be careful not to draw conclusions based on incomplete information, the perspective gathered early in the process can help plan subsequent data collection and analysis.

Robert Mager is one of the best known authors in performance assessment and instructional development.  In his book, Analyzing Performance Problems, he provides a process for analyzing a performance problem.  The first step in his process is to describe the performance problem.  While I agree that this is a prerequisite for a successful needs analysis, it is not always a fast or simple process.  It is often difficult to describe the actual problem but easy to describe the symptoms.  That is why it is critical to follow a systematic analytic process.  Since there are many approaches to needs analysis I will take several posts to describe how to conduct a successful needs assessment.

Next Page »