The SMART Lab
A framework for specific, measureable, achievable, relevant, and time-based performance goals
A framework for specific, measureable, achievable, relevant, and time-based performance goals
A guide exists to help lab managers use SMART goals to manage their own time,4 but not their lab or their direct reports. Preliminary work on lab-specific SMART goals is not widely available.5, 6
SMART goals are goals that are Specific, Measurable, Achievable, Relevant, and Time-Based.
Specific—What is measured? Is there a single key output?
Measurable—Is the goal quantitative? How is it measured? What are the units?
Achievable—Is the goal challenging? Is it attainable? Are sufficient resources available?
Relevant—Is the goal meaningful to the employees’ role? Is it aligned with the organization’s strategic objectives?
Time-Based—When will results occur? What is the timeline?
The SMART acronym gives managers a useful framework for developing goals. Considerable thought and effort are required, however, to develop goals that achieve their purpose of motivating desired, organization-critical behaviors. Poorly crafted or improperly executed goals may not succeed in motivating desired behavior and can even motivate undesirable or counterproductive behaviors. Managers need to carefully consider:
- Does the goal encourage the desired behavior?
- How could the goal be misunderstood?
- Could the goal encourage any undesired behaviors?
- How and when will employees know how they are doing or when the objective is fully achieved?
- Do the goal and associated metrics discriminate between top performers, average performers, and underperformers?
Harris discussed requirements for motivating desired behaviors in analytical chemistry teaching laboratories.7 The concept of discrimination is central to motivating desired behaviors. Harris defined discrimination as a measure of the difference between the best and worst performers, and found that discrimination correlated with student morale. Where discrimination is high, there is a strong correlation between achievement and assessment. Where goal discrimination is low, there is little apparent difference between top and bottom performers, leading to poor morale and underachievement.
Harris also discussed the importance of a rational and effective scale for evaluating the goal. Harris found that a five-point scale worked well, although in practice he extended this to include a possible score of zero. In the context of goal setting and evaluation, such a scale could be expressed as follows:
5—Exceeds all objectives
4—Exceeds most objectives
3—Meets performance objectives
2—Meets most objectives
1—Meets some objectives
0—Does not meet any objectives
QC-based SMART goals
Laboratories traditionally use customer satisfaction surveys to assess laboratory performance, with individual goals developed based on desired survey responses. There are a number of shortcomings with goals based on customer satisfaction surveys. Customer survey goals are more qualitative than quantitative, because the quality of the survey results depends on completion rates and timeliness of responses; this can be problematic even with supposedly mandatory surveys. Customer survey goals are subjective, with some customers evaluating the lab based on their ideal lab performance rather than agreed-upon service targets. Customer survey goals show poor discrimination, in that some customers are easily satisfied, whereas other customers are never satisfied. It is difficult to quantify individual contributions from customer satisfaction surveys. Finally, goals based on customer satisfaction surveys are reactive rather than proactive; it can take one year or more for the manager to get feedback and take appropriate action.
Laboratory managers can use data the lab already generates to develop SMART goals. QC (quality control) data is used to validate assay and laboratory performance, so goals based on QC data are obviously relevant. Goals based on QC data are proactive; analysts always know exactly how they are doing. Analysts can thus take immediate steps to improve, and there are no surprises or disagreements over progress, status, or assessment. Managers can track individual and group progress and proactively intervene as required.
Analytical laboratory key performance indicators typically include accuracy, precision, and turnaround time. Neglecting any of these three may lead to undesired behavior (e.g., increased turnaround at the expense of accuracy). Targets are based on external (to the laboratory), scientifically justifiable criteria. Supervisors’ individual goals are a composite of their personal scores for the above three goals with those of their direct reports, so supervisors are motivated to maximize team performance.
SMART accuracy goal
The accuracy goal is shown in Table 1. The key laboratory assay used PTC-1a8 CRM as a QC sample. Samples were assayed by aqua regia digestion with inductively coupled plasma (ICP) spectrometry finish. Although a suite of elements was typically reported, only Ni and Cu were used for the goal, as these were the key customer analytes. A competent, diligent analyst cannot do better than the 95 percent confidence interval for the CRM. A linear progression of the CRM 95 percent confidence interval was used. This is a stricter criterion than Harris used, and is also stricter than the ISO 17025 criterion of 100 ±10 percent recovery. Harris discussed that optimal discrimination is not necessarily obtained from a Gaussian distribution of scores, and that skewing the distribution may improve discrimination. In this case, one additional 95 percent confidence interval is added on the low side for Ni to account for mafic, nickel-bearing silicates incompletely decomposed by aqua regia that are present in the CRM but not present in the samples. Analysts were assessed daily on the average of the Ni and Cu scores for that day, with the annual score being an average of the daily scores. Each error the analyst authorized in the LIMS (laboratory information management system) resulted in a one-fifth penalty on their annual accuracy goal score.
| Score | % NI | % CU |
|---|---|---|
| PTC-1a Certified Values -> | 10.03 ± 0.07 | 13.51 ± 0.11 |
| 5 | 9.89 - 10.10 | 13.40 - 13.62 |
| 4 | 9.82 - 10.17 | 13.29 - 13.73 |
| 3 | 9.75 - 10.25 | 13.18 - 13.84 |
| 2 | 9.68 - 10.32 | 13.07 - 13.95 |
| 1 | 9.61 - 10.39 | 12.96 - 14.06 |
| 0 | Outside above range | Outside above range |
Specific—Based on QC samples’ agreement with CRM certificate of analysis.
Measurable—Percent of relative error.
Achievable—QC data combined with the above external, scientifically justifiable criteria were used to verify that the goal was suitably challenging (discriminating); the goal was modified (skewed) to account for a known, small negative bias for Ni, as discussed above.
Relevant—Customer assays are validated using this QC data.
Timely—Measured daily, with annual assessment.
SMART precision goal
The precision goal is shown in Table 2. Justification for the intervals in Table 2 was taken from Skoog, Holler, and Crouch,9 where they state for ICP that “under ideal conditions, reproducibilities of the order of 1% relative have been demonstrated … such high precision is not often achieved in the overall measurement process.” This statement referred to multichannel ICP, whereas the ICP used was sequential, so is likely to be of lower precision, especially considering the nature of the geochemical samples being determined and that analysts also prepared the samples they assayed.
Each batch of samples included one assay duplicate. Customers typically submitted samples consisting of nine different streams with varying concentrations/ ratios of Ni, Cu, and other elements. When developing this goal, precision was evaluated for each of the nine streams as well as overall. It was found that most imprecision was related to four of nine of the streams where Ni and/or Cu were present at low concentrations.
Over the course of the year, analysts’ large number of assay duplicates will average out between the lowand high-precision sample streams. Because supervisors assay a relatively small number of batches, it is possible that they may assay a disproportionate number of low-concentration assay duplicates. Supervisors’ individual annual scores were thus weighted to 4/9 if their ratio for these imprecise duplicate streams was outside the range of 1:2 to 5:4 to prevent them from being unfairly penalized if they happened to have a disproportionate number of low-concentration assay duplicates.
| Score | NI and CU Average % RSD |
|---|---|
| 5 | ≤ 1 |
| 4 | 1 - 1.5 |
| 3 | 1.5 - 2 |
| 2 | 2 - 2.5 |
| 1 | 2.5 - 3 |
| 0 | > 3 |
Specific—Based on agreement of assay duplicates.
Measurable—Percent of RSD.
Achievable—QC data combined with the above external, scientifically justifiable criteria were used to verify that the goal was suitably challenging (discriminating). Provision was made for the goal to be modified (skewed) for supervisors, as discussed above.
Relevant—Successful labs minimize variability.
Timely—Measured daily, with annual assessment.
SMART turnaround goal
The turnaround goal is shown in Table 3. Customer requirement for turnaround time was four hours from delivery to the lab. Turnaround time can be tracked in LIMS for each analyst.
Exceptions to meeting the required turnaround time were specified in the goal. Crew change-outs where no other analyst or supervisor was present to complete the work were exempted. Mandatory training and meetings that could not be rescheduled were exempted; otherwise, analysts would have to choose between a lower assessment or risk disciplinary action for missing meetings and training. Alarms when the building must be evacuated, as well as safety incidents, may make it impossible to assay the samples within four hours and were thus exempted; otherwise, analysts might be tempted to ignore alarms and safety incidents. Many analysts and one supervisor were members of the emergency response team (ERT) and were required to immediately respond to any alarm anywhere on the site. ERT callout within four hours of samples receipt was also exempt for ERT members; otherwise, lab staff would be motivated to quit the ERT.
Instrument malfunction and servicing were NOT exempted from the turnaround goal. Analysts were thus motivated to keep instruments well maintained to minimize downtime. Supervisors were motivated to ensure analysts were diligent with assigned maintenance, and to schedule maintenance and servicing to minimize customer impact.
| Score | Compliance |
|---|---|
| 5 | 95% |
| 4 | 90% |
| 3 | 85% |
| 2 | 75% |
| 1 | 50% |
| 0 | < 50% |
Specific—Based on time from sample receipt at the lab to the time the samples are authorized in LIMS.
Measurable—Time in hours, recorded in LIMS.
Achievable—Various exemptions, as well as what is specifically not exempted in order to motivate all desired behaviors, were detailed in the goal, and based on external criteria.
Relevant—Based on customer business requirement.
Timely—Measured daily, and assessed annually.
SMART goal concerns
Limit the number of goals. Focus, primacy, and impact decrease as the number of goals increases. Use the minimum number of goals required to motivate all organization-critical behaviors. If this results in more than a couple of goals, limit the number of target behaviors or use other means to motivate those desired behaviors.
Involve staff in goal development. They will be more than happy to tell you what they think is realistic and achievable, identify ways in which the goal may be misunderstood, and suggest negative behaviors that may be unintentionally motivated. Consulting staff during goal development decreases potential resistance to and resentment of the new goals and improves morale and engagement.
Use of laboratory data is not a panacea for developing SMART goals. Projects and nontechnical behaviors (e.g., safety) may not be amenable to the use of laboratory data. Other SMART metrics must be developed in such cases.
Not all desired behavior can be captured in a couple of SMART goals. SMART goals are just one tool to attain the organization’s goals. Other mechanisms—such as salary increases, recognition and awards, professional development, special assignments, promotions, gifts, praise, etc.—exist for motivating desired behaviors.
References
1. Fermilab Management Practices Seminar Quick Guide to Goal Setting, http://wdrs.fnal.gov/elr/goal_setting_2009.pdf, accessed February 4, 2015.
2. Goal Setting, http://www.stratishealth.org/documents/HITToolkitclinic/1.Adopt/1.2Plan/1.2HIT_Goal_Setting.doc , accessed February 4, 2015.
3. Guide to Goal Setting, Harvard University, Faculty of Arts & Sciences, http://hr.fas.harvard.edu/files/fas-hr/files/fy14_fas_guide_ to_goal_setting_9.23.13_v2.pdf, accessed February 4, 2015.
4. Generate More Time Using S.M.A.R.T. Goals, http://www.labmanager.com/management-tips/2011/02/generate-more-time-using-s-m-a-r-t-goals, accessed February 4, 2015.
5. K. Headrick. SMART Analytical Lab Employee Incentive Goals. 63rd Pittsburgh Conference and Exposition on Analytical Chemistry and Applied Spectroscopy, Orlando, FL, U.S.A., March 11–15, 2012; invited paper No. 650-4.
6. K. Headrick. SMART Laboratory Employee Goals. Chemical Industry Digest, Volume 26, August, pp. 86–90 (2013), invited paper.
7. Harris, W. E., Anal. Chem., 47, 1046A–1056A (1975).
8. PTC-1a Certificate URL: http://www.nrcan.gc.ca/minerals-metals/sites/www.nrcan.gc.ca.minerals-metals/files/files/pdf/tech-tech/PTC-1a-eng.pdf
9. Skoog, Douglas A., Holler, F. James, and Crouch, Stanley R., “Principles of Instrumental Analysis,” 6th ed., Thomson Brooks/Cole, Canada (2007), p. 261.