Simple Performance Management Tracking Template Here

The tool is intended to help you get a quick assessment of who might need coaching, encouragement, or focused training. It is a high-level look to pattern-match and quantify something that is generally extremely subjective. A byproduct of this template is you can use it on yourself as well!

Once you’ve taken a look and read on, please let me know what you think about these templates. I’m always looking to make them more meaningful.

Performance management is a big topic, but let’s stay tight in this particular instance and not worry too much about the details. Instead, let’s assess how we can get a quick understanding of how our team is doing day-to-day and over a longer span of time.

I use a similar tool when I am working through performance management, and found it to be highly effective.

Why This Is Helpful

Our brains are real jerks when it comes to thinking about something that happened months ago. When we don’t jot down our thoughts or notes from a day, we’ll likely forget anything that is not critical shortly thereafter. A major complaint when it comes to performance reviews (specifically when they’re done once or twice per year) is that managers lean into recency bias, forgetting what happened the first five months of the period and focusing only on the most recent month.

Recency bias is not intentional, it’s how our brains process information. Instead of relying on your brain to remember everything, what if you had an easy way to take note of the performance of your team day-to-day? The template explanation below aims to help with the quick note-taking of how performance is trending on a team.

Let’s dive in.

A Sliding Scale

Quantitative data around real-time performance is difficult in software engineering, especially the further into engineering leadership you go. What starts as very tactical (tickets completed, PRs reviewed/pushed, etc) turns to very subjective (are folks still engaged, are they inspired to do their work), so how can you keep track?

Put simply, you can boil every interaction down into a number. I like to use -2 through +2 in this case to identify neutrality and the extremes (let me know in the comments if you want further examples of what each number might look like).

-2 - This is really bad. The interaction was poor, and way under what I expected from the individual. Should be an outlier.

-1 - Not quite what you want to see, but maybe there was a lack of clarity or understanding. The interaction could be improved, but not the end of the world.

0 - Meets expectations. Nothing unusual in either direction and the individual was doing what was asked.

+1 - The interaction was better than expected and is a positive example of what going above and beyond looks like.

+2 - Blown away by the interaction, tremendous growth shown by the individual and unsustainable high performance.

As with all things, these numbers are subjective, BUT it starts to get us closer to quantifiable performance tracking. The goal is not to have all +2s forever, the goal is to skew toward the positive. Day-to-day that number may adjust and that’s totally okay. Focus on the averages. By week, by month, weighted, whatever it takes.

The Scale In Practice

Here’s a fake team I created for the template. On your team, you have six engineers: Brittany, Bob, Blake, Bill, Bella, and Brenda. These members are stored in columns, while the dates of work are in rows. For this example, we have a 7-day work week that everyone loves (there are never any complaints about working 56-hour weeks consistently, no way). I shouldn’t need to say this, but please don’t start a 7-day workweek at your company.

I’ve color-coded the cells with a min of -2 and a max of +2, to give a visual assessment of how someone is doing. From a quick look, without thinking about numbers, who is doing well (green or white) and who is not doing well (red).

Pretty quickly you can see you’ll probably need to work with Bill and Brenda. The pattern shows you that Bill may be pulling out of a negative few weeks, while Brenda is a bit more sporadic and starting to trend negative.

Blur your eyes a bit and look at the above. What else do you see? As a manager, you should be able to quickly determine any patterns (if you can’t that’s alright, it’s all about practice). My thoughts from the quickly generated random numbers:

• Bella is my most consistent performer. Steady at +1s, jumping into +2s, she’s probably someone who is nearing promotion or is a steadily high-performing individual. How does Bella continue to perform so well? Can you take that formula and share it with others?

• Bob is another consistent individual, minus two weeks from 12/14 - 12/22. What about that week was different? What might have happened and how did he recover so well from it? The recovery is likely a bright spot you can use for others as well.

• I’m worried about Bill and Brenda. Brenda is having a particularly rough December and Bill seems to be coming out of a rut. These are the two individuals I would be focusing on for performance management. Are they too far gone? Can they recover? What changed that they are not doing as well as previously?

What other questions do you get from the patterns above? Let me know in the comments below.

General Assessment

I’ve added some averages and further breakdown of the data (auto-updated as more numbers get added or changed).

Note I didn’t go so far as to break it into all of the numbers. I’m interested in an exact count here, but a general understanding of where the team sits.

• Do they have more “Above” interactions?

• What is the breakdown between categories?

I have 3 team performance scales in my mind that relate to the percentage breakdown of Below, Neutral, and Above.

This is not scientific, it is a gut reaction to the question, “what percentage of each category is shown by a High-performing, Solidly performing, and a Low-performing team?” Not exact by any means and each organization is different, but you can get a quick check for how the team is doing. Our 6B team is an interesting mix, but I would put them as “Solid” overall. More Above interactions than Below, but still with 33% Below.

An uneven balance like this shows me that the team fluctuates quite a bit and is inconsistent in their behaviors and likely their work as well.

Closing Thoughts

Every manager will use this tool differently, so I won’t go into extreme detail on how to use it. Instead, consider how you currently track patterns for your teams and individuals.

Where do you struggle with performance management currently? Let me know in the comments below or directly on LinkedIn!

If you don’t have a plan in place already, have no fear. I’ve got an easy-to-use template/reference here (Notion) and below I’ll go into way more detail on each part. This template is built from 10+ years of experience in engineering organizations of various sizes, witnessing good and bad on both the engineering and management side of things.

Template in Notion | Template in Google Docs

The very basics

Incident response can be extremely complicated, but at its core, here’s what you need to do.

1. Identify that it is an incident along with its priority

2. Start an incident

3. Stabilize the incident

4. Resolve the incident

5. Review the incident in a postmortem

You now hold the power in your hands, but before we dive a little deeper and get more detailed, here’s my guiding principle for running incidents in product engineering organizations: Fail With Transparency

Failing With Transparency

One of the guiding principles, when I’m thinking about running incidents in a product engineering org, is to fail with transparency. Too often engineering teams try to hide their mistakes, sweeping any bad bugs under the rug so other parts of the organization don’t worry about the software. I’ve found that you can sweep bugs and incidents under the rug, but if you’re looking to build trust with stakeholders, incidents give you a chance to cement the relationship.

Think of it like the status pages for AWS or the old-school Fail Whale with Twitter. Once folks know something is up, they really want to know what’s wrong and what’s being done to fix it.

Are stakeholders happy about incidents? Absolutely not, but they do expect incidents to occur from time to time. Instead of hiding, share your incidents, postmortems, next steps, and learnings. Pull your stakeholders in and call it what it is, an incident.

Starting with something simple like the following is totally fine:

This lets them follow along, inform their teams, create contingency plans, etc. Failing out in the open is better than trying to fix something before stakeholders notice. They’re going to find out one way or another, so why not make them part of the solution?

Also, when in doubt, over-communicate. Do this in a concise way, but give updates that everyone can understand even if the update is “no update at this time.”

Now that that’s out of the way, let’s take a look at each part of our template and tie it to the five points above.

Identify that it is an incident along with its priority

This is where it all starts. A monitor goes off, an alert is triggered, or someone sends a sad tweet saying they can’t get their free disc golf disc because the form is busted, you know the drill.

Problem Identification

Whatever problem was noted, you now must determine what is the real problem. Understand that this may shift as the incident continues, but don’t be fooled by easy answers. “The validation is broken on this form,” seems innocuous enough, but if you don’t uncover why the validation is truly broken, you may solve one piece of the problem, but not all of it. As the team begins to form, continue to question the core problem you are addressing.

The output of problem identification should be a Problem Statement. This is likely the only time you’ll see me link out to anything related to Six Sigma, but this is a great overview of writing a clear problem statement. Your problem statement will evolve as you learn more but use it as the guiding light for the incident.

Severity

Severity is an area of incidents where I like to follow the KISS method (Keep It Simple Stupid). No need to get cute or fancy here, use an increasing scale for decreasing priority. SEV1, SEV2, SEV3 should work just fine and the below table outlines the way Atlassian determines severity level for their incidents. No need to recreate the wheel unless it truly needs to be customized.

To say it a different way:

• SEV-1: Hold everything, this is a critical process that’s down and we should not move forward anywhere until this is resolved. There’s a big ol’ fire.

• SEV-2: Serious stuff, but the world’s not on fire. It will make folks grumpy and should be resolved by the end of the day. Contained fire, but could spread.

• SEV-3: Eh, bigger than a bug, but not super severe. You should probably fix this within a week if not within 2-3 days. Smoke, but nothing else.

Start an Incident

There are many ways to start an incident, whether through DataDog, PagerDuty, FreshService, or any other on-call SaaS. Whatever the triggering mechanism, or system you’re using, starting an incident involves making a broad declaration and shouting from the rooftops, “*cough cough* SOMETHING’S WRONG AND WE WANT TO FIX IT!”

My preferred method of starting an incident (when remote) is to get all the appropriate parties in a channel and onto a call. Defining “the appropriate parties” can be broken down thusly:

1. Who’s going to fix it?

2. Who cares that it gets fixed?

3. Who can identify impact?

4. Who fixed it last time (or knows about the area that needs fixin’)?

Those four questions can usually get you who you need to get to work on an incident. With your crew in tow, you may now move about the incident. Think of it as a very boring (or exciting to some) way of forming The Fellowship of the Ring or Dom Toretto assembling a team for whatever the plot is for Fast & The Furious 15.

Once the team is assembled, you can fill out the first major portion of the template.

Incident Management Team aka “The Crew” (with explanations)

Incident Commander

Who is handling the communication for this incident? As incident commander, you are pulling folks together and ensuring folks stay on task. You are also responsible for keeping stakeholders and incident-watchers up-to-date. This is especially critical in SEV-1 and SEV-2 incidents. I like to ensure there is an update every 15 minutes or so in the early goings of an incident, providing clarity about what is being tried, what was identified, and how the team is proceeding.

This role is also critical as a facilitator. View yourself less as a “hey can I get an update” requester, but as the captain of the ship, steering conversation if it veers off course, or asking “dumb” questions to open new lines of thinking. This is especially helpful when stuck. Some questions to consider:

• What changed recently?

• What is the simplest reason this might not be working anymore?

• Any third-party vendors we’re relying on for this work?

• What happens if we reset/revert?

I put “dumb” in quotes for a reason. It isn’t that these are bad questions, they’re just obvious. Problem is, when an incident is going on, sometimes individuals forget to think through the obvious and hop straight to more complex complications. Helping the team slow down and think about the simplest solution can go a long way.

Remember Occam’s Razor, paraphrasing but, “the simplest reason is usually the right one.” Essentially, start small and work big with dependencies and complexities, eliminating reasons along the way.

Technical Lead

The Technical Lead for the incident is the individual, typically an engineer, that is leading the technical resolution of the incident itself. Whether a subject matter expert in the area or simply the first to respond to the incident, the Technical Lead tends to make final decisions on what order solutions are tried.

If no one is signing up for the role, the incident commander should assign it to whoever appears to be taking the lead role at the moment. This is not intended to add additional pressure, but to identify one individual that can provide updates and make decisions throughout the remainder of the incident.

Impact Assessor

Assessing the impact or “blast radius” as I and others like to call it, is key at the beginning of an incident. Blast radius refers to how broad the issue is and how many individuals it affects.

The Impact Assessor is the number cruncher, trying to determine the overall cost or opportunity cost of the incident itself. They will likely, or should, ask questions like:

• How many users is this affecting?

• What dollars or time lost can we associate with the area that is down?

• Where can we get the data from? (database, dashboard, etc)

• What will the impact continue to be if not resolved?

This can be an engineer, product manager, engineering manager, data analyst, or some other party not focused on resolving the problem.

Problem Resolvers

Anyone who is actively working on the resolution to the incident. Whether through scripts, code updates, or reverts, whoever has their hands on the keyboard goes on this list. These tend to be the individuals that are more heads-down and focused during the incident.

The expectation for these individuals is to divvy up work (if appropriate), talk through solutions, and reconvene as needed to work through the next steps. For a SEV-1 incident, I typically like everyone to stay on the call until the incident is stable, but give space to the engineers that need some quiet to fix the problem.

Set a check-in time, usually based on how often updates are needed for stakeholders, and agree on how you will communicate through the silence. The worst thing you can have in an incident is silence that no one understands. When quiet, bystanders and stakeholders will assume the worst and wonder if anyone cares or if anyone is working on the issue.

Stabilize the Incident

Once started, the immediate goal is not to resolve, the immediate goal is to stabilize. Think about your incident as a leaky bucket. As you put water in for the day, you notice the bucket has one or several leaks. Your first thought is generally not, “how do I rebuild this entire bucket so the leaks stop,” but instead should be “how do I stop these leaks?”

More specifically, if the incident is preventing all of your customers from checking out, the first goal is to get them checking out again. Once you stabilize the flow, you can begin working on the root cause and solving the larger issue. You can think of it this way:

Keep the team focused on the solutions that will bring about the fastest stabilization, but understand how to weight them appropriately. If there are two stabilization solutions with one taking fifteen minutes and the other taking twenty minutes, understand what additional benefit the twenty-minute solution provides and how costly those five minutes will be.

Look at your monitors or alerts to determine that the incident is indeed stable, mark it as such, update relevant parties, and move on to the Resolution phase.

Resolve the Incident

In some cases, you can skip this section entirely because the stabilization was also the resolution, but even in those cases an important question to ask is, “how do we prevent this from happening again?”

It’s a question that may be better served in the postmortem, but if the stabilization is not going to last for days at a time, the team must determine a more steadfast solution. If nothing else, it is likely helpful to reassess the monitors or trigger points to determine if they were loud enough or too loud.

In cases where the stabilization was a small patch, continue working as an incident team to determine the long-term fix. Any additional logic that needs to be sorted? A broader sweep through the codebase to eliminate the opportunity for this to happen again? There are plenty of ways you might need to resolve the incident, but the point of stabilizing first is it gives everyone a chance to breathe. By taking a moment to look at the broader picture, you can see what fringe areas may be affected, additional avenues that need assessing, or more tests that need to be written.

Resolving is essentially putting a stamp on the incident saying, “This incident is stable, as complete as we expect it to be in the short term, and we don’t expect to see this again.” The resolution allows all watching parties to go back to their own work and stop worrying about the incident.

Template in Notion | Template in Google Docs

Review the Incident in a Postmortem

This is probably a whole post on its own, but each incident is not officially over until the team holds a postmortem to review what happened. Postmortems are extremely common throughout not just the tech world, but globally in organizations. For a good primer in postmortems, check out PagerDuty’s overview.

In the template I’ve provided, you can continue using the same document for the postmortem. Key items to review:

1. Final Impact

2. Detection Method

3. Steps used to resolve / Overall Timeline

4. Root Cause Analysis

5. Action Items for further prevention

Final Impact

With the incident resolved, the team should have a good understanding of how broad the impact was, who was impacted, and what the associated costs are with the impact. Ensure your stakeholders have all of their impact questions answered, as they will need to surface the impact to their teams and leaders. Use the individual designated as the Impact Assessor to finish these details. It is also helpful if you or the team can tie the impact back to “what does this mean for the business?” but that is of secondary importance outside of the primary impact.

Detection Method

How did we know there was an incident? Use two primary categories, manual or automated, for the detection method.

• If detected manually, is that a stakeholder or customer that said something was broken? Or did we accidentally stumble upon it ourselves?

• For automated detection, was it an early alert or did we only get alerted once the incident got real bad?

  • Secondary for automated would be, did anyone notice the automated alert or do we need to increase the volume of those alerts?

  • Do we need to reset expectations about how to identify automated alerts?

All of these are great questions about detection methods and how you might improve for future incidents.

Steps Used to Resolve / Overall Timeline

Review the timeline and the steps used to resolve the incident and ask some of the following questions:

• What went well

• What didn’t go so well

• What did we miss initially that could have helped us resolve faster

• What will we change next time

• Do we feel the response time was adequate

If you’re using toolsets from PagerDuty or DataDog for your incidents it is likely that you have a full timeline built out from Slack or your company’s chat tool. If not, be sure the incident commander is taking good notes during the incident for review.

Fill in any gaps as you review with the team. During a postmortem, no detail is too small. You are attempting to paint the full picture of the incident to improve for the future.

Root Cause Analysis

How you get to the root cause is up to you and the team, but dig deep. The root cause is unlikely to be a simple answer, so spend at least five minutes trying to understand if it’s a people, process, or program problem.

The Five Whys is typically used as the golden standard for root cause analysis (RCA), here’s a good overview from Kanbanize. At its core, the Five Whys prods you to keep uncovering layers until you get to the true root of the problem. Explore the root and explore solutions for the root problem before moving on in the postmortem.

Action Items for further prevention

Show me a postmortem with no action items and I’ll show you a team that hasn’t thought critically enough about the problem. There is always something to do after an incident. Whether it is additional research into the problem area, assessment of triggers and alerts, or whole projects dedicated to refactoring the problem area, there are always action items. Determine who will take the action items and the expected timeframe to complete them.

Bonus: Remember to keep it blameless

Regardless of how many times we’ve been through an incident or postmortem, I like to remind the team that the postmortem is intended to be blameless. In other words, no finger-pointing allowed. We are all working against the problem together. Remind the team of this at the beginning of the retro and remind them again if blame is getting assigned to an individual or group.

Do not tolerate blame assignment.

Tips For Managers During Incidents

Have a plan in place

By reading this guide you’re already further ahead than most organizations. Please do not be a company that says, “well, if we don’t talk about it, then we’ll never have an incident,” as if it were some weird superstition. Cliche quote incoming but:

The first version of your incident response plan does not, and likely should not be the last version. After several incidents using the framework, hold a retrospective with the teams that took part and identify what went well, what didn’t go well, and what confused them. Make some changes, communicate those changes and keep on tweaking.

With a plan in place, you can now set clear expectations about how to run incidents and what you expect from each member of your team. By setting clear expectations, you can ensure each incident is moving forward appropriately, or measure and review when things are off. Reset expectations as you go if needed, but continue working with your team to help them through incidents.

Especially in the early goings I like to be in most of the incidents my team is on so I can lend a helping hand on the facilitation side. This might take the form of being the Incident Commander, or just riding along, but either way, it helps you set the behaviors you want to see continue. It’s also a good way to test the process you helped build to see what feels off.

Have a fast build pipeline, or a way to sneak around it

Unpopular opinion, but I don’t want all of my tests to run if I’m trying to prevent $2 million from leaking out of the business. I need a way to get the fix in there now and not ten minutes from now. While build pipelines are created to help with resiliency of a codebase and not letting bugs creep in, there must be a way to get hotfixes in quickly.

If your codebase and pipeline don’t currently allow for this, work with your teams to figure out how you get there. There’s nothing worse than having a solution available to stop the leaking, then watching CircleCI buffer for five to twenty minutes.

Be Helpful, Not Helicopter

Check in with the Incident Commander to see if you can be helpful at all. If they say no, stay out of the way. Watch the incident and offer suggestions or thoughts where it makes sense, but if you’re not a main contributor, hold off.

Being a helicopter and hovering or constantly asking for updates is a great way to distract the team from solving the incident. If you absolutely feel like you must be involved, spend more time on the data assessment or stakeholder communications. I’d also ask that you take a good look at why you feel you must be involved, and sort that out so you aren’t a bottleneck in the future.

No swooping, no pooping, and no seagull managers during incidents. Let your people work.

Know how incidents impact work and productivity the 24 hours after

Most organizations with on-call rotations or folks who have to hop into incidents don’t give any additional incentive for doing so. It tends to be “part of the job” which is fine! Some companies pay overtime for these sorts of situations, but I’ve never been in a company where that is true.

As a manager, don’t be tone-deaf about the effects of incidents in the following 24 hours. There are follow-ups, next steps for resolution or clean-up, and likely more meetings to discuss further. Give that individual space and for the love of all things, don’t expect them to be on your 9:00am standup if they were up at 4:30am to resolve the incident. More importantly, tell them that. Make it the norm that those on incidents overnight are given leeway as long as they communicate what to expect.

If you’re looking for your engineers to burn out quickly, force them to be at all meetings and on time the day following an overnight incident.

Run an incident before you need the Incident Response

As much as you want to believe that everyone read the documents and understands them, have a gameday where you run through a fake incident. Schedule time for everyone to run through the appropriate roles from identification to resolution (you can skip the postmortem). Note: Be sure you let your stakeholders know you are running a test to not spook them.

This sets a comfort level with the tools and helps you understand where your team might falter a bit in a real incident. True incidents don’t have the luxury of looking around with the toolset to try and remember which buttons create which part of the incident response.

Keep calm, and be a calming presence

Which group do you want to work on a problem with, one where everyone is yelling and has their hair on fire? Especially as a manager, you must maintain the calm and cool things down if they get heated. There’s no sense in getting all riled up in these conversations because it’s not going to help anyone.

Honestly, it’s also okay to be a little silly during an incident. Just because the work is serious doesn’t mean everyone’s demeanor needs to be serious as well. This is not to say it’s time for someone to bust out the joke book or try out their tight five, but it’s alright to keep things lighthearted. Don’t force this by any means, but don’t shy away from it if that’s the personality of your product engineering team.

Tips For Handling Incidents In A Product Engineering Org

When I think about how I typically handle incidents, the following comes to mind. Not all tips will be applicable for every incident, but each boils down to:

1. Fail with transparency

2. Show you are working the problem and care. This is more just putting on paper what I’ve done in the past and is not commentary on recently handled incidents.

3. Pulling in the right folks initially

   1. Use domain documentation, broad slack channels, whatever makes the most sense

4. Stakeholder awareness is top priority.

   1. Helps stem the bleeding, and while sometimes awkward for them to see how the sausage is made, it’s better than staying quiet

   2. Pull in the core stakeholders at the top of the hierarchy so they can appropriately communicate with their folks

5. Get engineers to stabilize, then work on root cause

   1. Eliminate what it’s not, low-hanging fruit, things that it would be stupid if it was causing the issue, Occam’s Razor, etc, etc

6. Updates every 15-30 minutes, even if the update is “there is no update” or as progress is being made

   1. Helps stakeholders and others know that items are still being worked on

   2. Helps the timeline for postmortems

7. If pausing on work, clearly lay out the next steps for the next day

8. Hold all conversations in the incident channel

   1. DM’s are the enemy! Easy to lose track of who is saying what and who has what information if it’s not done in the open

9. Being clear about impact or possible impact

   1. Determine if additional data is required from a team outside of your scope

10. Link impact/outage to cost wherever possible

11. Next day follow-up

    1. Is the incident still occurring? How do we know it’s stopped if stable?

    2. If fixed, do we need support to resolve anything outstanding?

12. Letting your leader know

    1. I generally let my boss know severity, how much chaos it’s causing, and overall temperature as soon as it starts

13. Ticket creation for appropriate teams to ensure we are tracking the work somewhere

Template in Notion | Template in Google Docs

Checklist For During the Incident

• [ ] Create a channel for the incident

• [ ] Pull in the appropriate subject matter experts (or on call folks if after hours)

• [ ] Pull in the appropriate stakeholders

• [ ] Let your leaders know (may be in the stakeholder group already)

• [ ] State the issue clearly along with current state and what that means for those affected

• [ ] Identify root cause or eliminate what it is not if the root cause is elusive

• [ ] Submit updates every 15-30 minutes, even if the update is “there is no update” or as progress is being made

• [ ] If pausing on work (or incident is stable), clearly lay out the next steps for the next day

• [ ] Hold all conversations in the incident channel (avoid DM’s)

• [ ] Be clear about impact or possible impact

• [ ] Determine if additional data is required from a team outside of our scope

• [ ] Link impact/outage to cost wherever possible

• [ ] Next day follow-up

  • [ ] Is the incident still occurring? How do we know it’s stopped if stable?

  • [ ] If fixed, do we need support to resolve anything outstanding?

• [ ] Ticket creation for appropriate teams to ensure we are tracking the work somewhere

A major thing to remember within any interview process is that each portion of the pipeline should support the next steps. If the initial screen shows they may be weak in Ruby on Rails, the Technical Screen should either validate or disprove that they are weak within that area. If the Technical Assessment gives some mini flags of ego or cockiness, the Values Fit should work on validating or disproving that, and so on.

The simple template looks like this:

• Initial Screen (45 minutes max)

• Technical Skills Assessment (1–2 hours)

• Values Fit (1 hour max)

• Final Check (30 minutes)

The total time from the above is roughly 3-4.5 hours. Might feel short, but I’ve found great success in this amount of time.

I prefer to break this into 2 days:

• Day 1

  • Initial Screen

• Day 2

  • Technical Assessment

  • Values Fit

  • Final Check

• By Day 5

  • Offer or Rejection

Be respectful of a candidate’s time and remember that they will likely need to take PTO in order to accommodate the second day of the process. Don’t get all high and mighty and say “Well, if they don’t want to spend time with us, they must not want the job!” Have some empathy and work with them to make it work.

Hiring Considerations

Before we dive into each section, I want to outline the variables associated with shortening or lengthening an interview process. A while back I was asked to pour jet fuel on a process, which was an incredibly fun task to think through. In doing so the following variables came to mind as factors that determine how fast a company can really go.

1. Hiring Accuracy - What percentage of individuals must be retained (voluntarily or involuntarily) after X period of time

2. Speed to Fire - Does our management group have enough discipline and time for proper performance management, and are we okay cutting ties quickly

3. Speed to Productivity - How quickly do we need these individuals to be productive on their team or teams

4. Available Support - Are our mentor-capable engineers available to support their new team members or do we have other structures in place to support them

5. Level flexibility - Asking for ten junior engineers is much easier than five staff engineers. Are there gains if you bring in juniors quickly vs staff slowly?

6. Location flexibility - If the company is fully remote, it’s much easier to attract talent at different skill levels because the pool is much broader. Needing folks to relocate or convincing them to commute will likely drag your process out.

Core metrics you should always track for hiring:

• Cost to hire: From resourcing to the moment they start, what is the total investment you have put into that individual? It’s not only the cost of one individual’s path, but the path of all others that didn’t make it that far. The total cost of recruiting, interviewing, etc, divided by the number of hires = cost to hire.

• Speed to hire: There are two flavors of speed. The first is time to close the open role, or how long it takes from when you make a role public to that person’s first day. The second is time from first contact (cold email, LinkedIn Recruiter message, etc) to the time they start. Note, for each of these it goes until the person starts, not when the offer is signed. You can go with the signed offer as the end date, but I’ve seen too much weirdness happen post-signing to pat myself on the back at that point.

Now that we’ve covered the hiring considerations, let’s dive into the parts that make up our hiring process.

Subscribe now

1. Initial Screen (45 minutes max)

Key Goal: Early assessment as to whether they seem to be a good technical and value fit for the company and team.
Who conducts: Hiring manager, or Technical Recruiter

Summary:

• Candidate Background/Overview

• Company/Role Background/Overview

• Quick technical check

• Candidate Questions

• Additional Company Questions

The initial screen is equal parts of you selling the company to the individual and the individual pitching themselves to you.

Candidate Background/Overview: Start off by asking them to give a bit of their background, summarizing portions of their career that make them a good fit for your company. By this point you should have their resume, so don’t ask them to give you a chronological breakdown of every position they’ve held. Highlighting the roles that transfer best into the new role should give you a good sense of if they’re a one-skill master, or if they’re well-rounded and capable of picking up new skills.

Company/Role Background/Overview: Once they’ve gone through their background, ask if they have any initial questions about the role or the company. If they’ve done their homework, they may have several right away. If not, no worries, dive into your overview and pitch. At this point you have likely given this pitch many times, so be sure to ask if they have any additional questions or clarifications once you’re finished.

Check all the basic boxes. Do they have experience with the technology or subject matter you and your team work with day-to-day? If not, are they willing to learn and do you believe they can learn? Are their skills and prior work transferrable?

Give them a chance to ask any additional questions on their end. I like to close with, “Anything additional you would like us to know about you?”

Finally, if you feel the candidate fits 70%+ of the criteria for the role, discuss the next steps and check on their availability for the next portion. If you feel they do not, you can do one of two things.

• Option 1 - Let them know you will review with the team and get back to them in a few days on next steps. Don’t let this drag out and absolutely do not ghost them. If they’ve spent any time with you, you owe it to them to give them some resolution.

• Option 2 - Tell them on the call that based on what you’ve heard you don’t feel they are the right fit and you’ll be ending the process there. This is more of a gut punch, but it’s also clear and direct. I’ve done this a few times, but also added what they should look into for additional training (especially with less experienced engineers), and offered myself up as a resource if they had additional questions. It hurts, but I’ve established some good relationships this way and even hired a few later on in their careers.

2. Technical Skills (1–2 hours)

Key goal: Assess problem-solving and technical skills, as well as ability to work with others on the team.

Who conducts: Engineers, preferably from the team that is hiring, varying levels. Don’t make the mistake of only seniors interviewing for senior+ roles. Varying levels will work with the individual, so their opinions will likely differ.

The technical skill assessment gives you a snapshot of how an individual might work in your environment and with your engineers. There are a few different types of technical assessments that can be considered. Decisions here will depend on the six hiring considerations above and the speed at which you would like to move. Below are my three optimal options.

Home-rolled code day:

My preferred method of technical assessment if possible, the home-rolled code day allows you to create a custom 2-hour interview where you can make any code specific to the job they will do if hired. This method has the benefit of giving you an idea of their technical chops in your stack, but also gives them an idea of what the codebase or work will look like day-to-day.

The biggest challenge with this method is finding a problem that can be solved in a short amount of time, is specific to your company’s work (without needing to be production ready), while also giving the interviewee enough of a chance to understand the technical stack they might work in. Consider putting the files up on GitHub, that way the individual can pull the codebase down and test their environment ahead of time. Environment problems are not a reason to fail a candidate.

This method has the largest up-front cost and will require some engineering time to build initially, but the payout is huge for your process. It looks like this: one-hour technical challenge that represents real work, one-hour debugging or architectural discussion that represents real work, depending on company need. Each session should also be viewed as pairing with the individuals interviewing the candidate.

Homework, pre-work, and live review:

When we were testing for Front End engineers at a previous company, we would send homework to candidates to weed out candidates at the beginning of our process. If you go this route, the exercise should be no more than an hour, and you should know what red flags you are looking out for. Dogfood your process to make sure it is no longer than an hour, instead of stating it. While I generally don’t like homework for interviewing, there are a few roles where it is beneficial.

After the homework portion, we would bring individuals in for a few hours to sit in the office and work through another UI problem (with a simple API and some UI to sit on top of it). This represented real work they would do day-to-day, but gave them focus time to get it done. We were also non-remote at the time, so it gave them a chance to work in our environment. It is easy to give this time in a remote environment as well. Be available for questions during this time and note check-in points where you will see if they have any questions for you. This portion of the process can be telling about whether they will reach out if they get stuck (there weren’t any “gotchas”). While most individuals were able to get the work done, or ask questions along the way, there were definitely anomaly candidates that this process sifted out.

The last piece of the interview was a code review with the individual along with a few prompts about how they would extend the code further if needed. Completion was not the only way to pass, but was a good sign of speed or expertise with the toolset. Discussion, ability to take feedback, and ability to think through future states told us more about a candidate.

Outsourcing:

There are several companies that specialize in outsourced technical code tests. Companies that come to mind are Woven, Byteboard, and Karat, each giving you the ability to outsource and standardize testing immediately. If speed is your top priority, outsourcing the assessment is your best bet.

Each of the companies mentioned gives different levels of interviews in different languages. Each also has consistent interviews with set scoring to help eliminate bias in the process. It also removes any internal concerns of sandbagging or sabotage.

It might feel more impersonal, but if it saves you and the candidates time (they can take the test whenever is best for them), is it really a negative?

There are plenty of other ways to assess technical skill, but the above methods are what I have found best when trying to stick to a 2-hour timeframe. What are some other methods you think work well?

Note: The technical assessment should not be a fishbowl process. Especially if you work in an engineering culture that promotes collaboration, the interview should also be collaborative. The interviewee is not there to put on a show, they are there to get a sense of if they would be comfortable working in an environment with your engineering team. I’ve seen too many technical interviews where a good engineer panics because they are getting nothing from the interviewers.

3. Values Fit (1 hour max)

Key goal: Does the individual align to the company and team values or are there red flags that prove otherwise.

Who conducts: Cross-functional partners, stakeholders, other engineers

The Values Fit portion of the interview is debatable for me. If you’ve included questions about your values as an organization and as an engineering team up until this point, you may be able to skip over this portion. 

Where I see the biggest benefit of this particular interview is the cross-functional partners. Product Managers, Designers, or engineers from other teams can give a different perspective of the candidate. Where the values interview becomes difficult is training. The more individuals you add to a process, the higher the risk of unconscious or even conscious bias leaking into the process. More training is needed as is a review of scores from interviewers to ensure they are assessing on the right areas.

Keep in mind that it should be made very clear how this interview differs from the others. Candidates are already in a fog during interview day, so clarifying that they will be asked more about how they work with others, past experiences, and optimal work environment can relieve some stress of context-switching.

Another way to think about this interview is the “soft skills” or “human skills” section, where the technical was the “hard skills.” This portion of the interview is where many companies dive into the “Tell Me About A Time…” questions. While these questions are helpful, I have a few that tend to tell me more about an individual. I go into detail here, but the four questions are:

1. What was the last piece of feedback you received?

2. What was the last piece of feedback you gave?

3. What project are you most proud of?

4. What’s a project that didn’t go so well?

Themes that should be answered by the end of this interview are:

• What are they like to work with?

• Would I be able to collaborate with this individual?

• Do they fit the style of work we are trying to do? If not, what concerns do I have about that?

Where there are concerns, did the interviewers give a chance for the interviewee to clarify or follow up? Feedback in this section holds less weight in the broader interview process, but can be beneficial for any red flags or patterns that emerge in other portions.

An example where this was helpful. A candidate nailed the first technical portion of the interview but in the second part of the technical assessment and the values fit interview, they were condescending toward all female interviewers. One instance of this is generally enough to fail a candidate, but when it is across multiple interviewers it is even more clear.

4. Final Check (30 minutes)

Key goal: Answer any remaining questions on either side.

Who conducts: Hiring manager

At the end of Day 2, the hiring manager should spend time with the candidate to see how things have gone. By this point, the Hiring Manager will likely see some of the feedback coming in and can also ask the interviewers to see if there are additional areas to dig in.

I generally take this time to see how the process has been for the candidate so far and if they have any questions about the company, the role, or the teams. This is also a good time to validate that the job still looks like something they’re interested in. In some cases it’s not and you can have an open conversation about their concerns.

Questions during this portion should be about what makes them nervous about the potential new role, areas where they think they would need support, or how their previous experiences would help them transition into the position. I also like to ask about how they like to be managed, how their best/worst managers managed them, and what they would be looking for from me if they took the role.

This is a final check because both candidate and interviewer should feel like they know where things stand at the end of the day. If things have not gone well or you are confident it will be a rejection, it’s okay to have that conversation.

Before closing, explain the next steps:

“Thanks for your time today. I’ll be getting with the team to review all feedback and you should hear something from myself or RECRUITER in the next three days. During that time, let us know if you have any questions or concerns.”

Be quick with the review and follow-up to ensure you are doing right by the candidate and keeping your process moving. Personally, I like to finish all of it in the following 48 hours if at all possible. Dragging it out past 48 hours is tough both internally and for the candidate.

5. Offer or Rejection (30 minutes)

Key goal: Present the result of the interview process.

Who conducts: Hiring manager and/or Technical recruiter

Unlike L’Orchestra Cinematique’s song in the trailer for Edge Of Tomorrow, this is the end…of the process. At this point you reviewed all feedback, had follow-up conversations where needed and determined whether you would like for the candidate to work with you and your team.

If you are extending an offer, you met with your internal team and know what will go into the offer from a salary and benefits perspective. My quick take is if the individual has stated a desired comp already, either hit it or go above what they asked for. I generally expect folks will come back with a counter (it’s a good practice as a candidate), but never lowball their initial ask unless they oversold their skills or you don’t mind them rejecting the offer. Keep in mind if you do undercut their initial stated comp request, your probability of losing the candidate skyrockets. Common sense, but lowballing is a bad way to start a relationship.

If you are rejecting the candidate, be clear about why and make clear whether there is an opportunity for them to take a position in the future. If they are a good team fit, but their skills do not match, let them know how the may enhance those skills for the future. They may tell you to shove it, but rejections with no reasoning are harder to swallow. If the candidate is being rejected because they are a total jerk, it may be more difficult to find the right words, but whatever you choose they will always see you as wrong, terrible, and incompetent.

At this point, the hope is you have closed the deal or cut the candidate loose and can move forward accordingly. Regardless of the outcome, take a moment to look back at the process for that candidate and see if there are any adjustments you should make.

No reference checks

Simply put, reference checks are a waste of time if you’re trying to determine whether the individual will be successful or not. I’ve never learned anything in a reference check that I didn’t already know or was revelatory. The key thing to remember, no one will give you reference information if they’re not prepared to say something complimentary about the individual. 

Remember, you are not some incredible detective that will shine a bright light on something bad the person did by interrogating a source provided by the individual.

I’m sure there are certain scenarios where this can be impactful, but I’ve yet to find them.

No Brain Trust Reviews

I’ve worked in several places where all individuals involved in the interview process for a candidate get into a room and discuss whether the person should be hired or not. I’ve also eliminated that process at all of those companies because it is a waste of time and unproductive. Here’s why:

1. The loudest voice wins

2. The quieter voices don’t have a seat

3. There is rarely anything new to learn

Group-think is all too common in these scenarios and whoever casts the first stone ends up leading the decision in the room for the other individuals. Descenting opinions are typically difficult to express, and any positives or negatives for a candidate tend to get buried if one interviewer is loud about their stance.

It is more beneficial to spend time with individuals to clarify their feedback or ask additional questions. Do not ask leading questions when following up, but see if they can expand on why something seemed like a positive or a negative. We all see things differently, so where I may view something as a positive, you may view it as a negative or vice versa. Chatting through the difference in opinion is better served in a one-on-one setting than in a group conversation.

Final Thoughts

Whew, there you have it. A small template with a lot crammed into it. The key thing in any interview process is that it is not a crockpot. You cannot set it and forget it. The process is a living thing that should evolve and improve over time, shifting based on hiring considerations and what roles are needed and when.

Keep poking at it and review metrics frequently. What else do you think about for your software interviewing process? What did I miss? Let me know below!

Get the template here: RICE Scoring Example [Make a Copy]

Is this a perfect system? Absolutely not. Is this helpful for testing quick assumptions and getting some prioritization on paper? Absolutely yes. To break it down simply, RICE scoring assesses Reach, Impact, Confidence, and Effort to get to a final score which can then be compared against other project scores.

As with other templates, I like to remove numbers from the initial assessment. Instead, I like to use words that release data-minded folks from the shackles of being too analytical too early. Simply stated, RICE Scoring is like deciding on recipes for an upcoming family meal you’re responsible for cooking:

• Reach is like the number of people you're serving. The more people, the higher the reach score.

• Impact is like the taste/nutrition of your dish (depending on what your guests value the most). How much will this meal benefit your guests? The bigger the impact, the higher the score.

• Confidence is like how difficult the recipe is. How sure are you that you can deliver this meal successfully? The more confident you are, the higher the score.

• Effort is like the time and effort required to cook your dish. How much work will it take to cook this meal? The less effort required, the higher the score.

The template itself is adjustable for both words and their weight (numerical value) in the “Data” tab, so change as is appropriate for you and your teams. Associating the categories into a final score is quite simple and boils down to the following formula:

Simple, made even more simple from the spreadsheet which will convert everything into numbers for you automatically. Now that you have a sense of what RICE Scoring is, let’s take a look at each section.

Reach Explained

When we talk about Reach in RICE scoring, we’re talking about the size of the audience we expect the project to affect. If you have an exact number, great, otherwise a rough approximation will do.

Something helpful, especially in early stages, is to think about your reach number in exponential quantities to help generalize the reach itself.

• Small = 10 people

• Medium = 100 people

• Large = 500 people

• Huge = 1000 people

You can adjust those numbers to whatever is most meaningful for you and your teams in the “Data” tab of the spreadsheet.

As long as there is some gap between the weight given to each category, it will be a meaningful data point.

Impact Explained

I deviate a bit here from the standard Impact scoring, leaning into a 5 as the largest impact and 0.5 as the smallest. Honestly, if the impact is less than 0.5, you’re probably never going to spend time or resources on that project anyway.

The numbers and terms for Impact assess the influence on the objective. Said a different way, Impact can, and should, reflect a quantitative goal of some sort and the impact of the feature on that goal.

In the template, I’ve got it broken down into the following:

• Minimal = 0.5

• Small = 1

• Medium = 2

• Large = 3

• Mega = 5

A goal in trying to identify the Impact is understanding the goal of the feature itself and your proposed metrics. Something like the Impact score becomes a forcing function of getting tied to the proposed value and what outcomes you expect to achieve with the project itself.

Confidence Explained

How confident are you of the Impact you are aiming for? The higher the confidence, the higher your score will be. The lower the confidence, the lower the score will be. Makes sense overall. For this spreadsheet, I’ve noted the following categories: Unclear, Low, Medium, High, and Highest. These categories translate into percentages to use in the equation.

• Unclear = 20% confidence = 0.2

• Low = 40% confidence = 0.4

• Medium = 60% confidence = 0.6

• High = 80% confidence = 0.8

• Highest = 100% confidence = 1

Go with your gut on this one. You should have a sense of how much of a moonshot the project is, and your confidence level should reflect as much. Think of it as confidence in the proposed Impact of the project, combined with confidence in the effort it will take to make it happen.

Effort Explained

Remind your teams, this is purely an estimate. Once it is determined that you will do the work, deeper consideration should be made to the breakdown of the work itself. For RICE Scoring purposes, I like to break it down by weeks of work.

• EZPZ = 0.5 weeks

• Small Lift = 1 weeks

• Mid-size = 2 weeks

• Hefty Chonk = 4 weeks

• Herculean = 8 weeks

If you’re laughing at me saying 8 weeks is a long time, consider that every week is critical to moving the business forward. Even if you’re not in the sort of environment that pushes that sort of thinking, treat each week as a sacred unit of time and do what you can to make it as impactful as possible.

Subscribe now

Other Considerations

I’ve worked in highly regulated industries (medical and insurance) and understand there are some items that you must do regardless of how helpful the project will be for folks. Don’t get trapped by thinking Impact and Reach are only for customers though, those assessments could also be indicators of protecting the company legally and the broad strokes of Impact there.

Set timeframes to revisit the data. The scoring itself should be reflective of a period of time you expect for the impact and reach, but continue to revisit over time to see if your assessments were correct.

Outside of those two considerations, remember that RICE Scoring is not a perfect system. It is intended to be a quick way to understand what you do and don’t know and how priorities stack based on that information.

How else have you used RICE Scoring in the past? Let me know below! Or, heck, tell me how I’m using this wrong so I can get better.

I implore you, please do at least, the bare minimum and fill out this simple meeting agenda. We’ll cover what each section entails further down.

Behold, the simplest meeting agenda template ever!

Super Simple Meeting Agenda Template (copy/paste)

1. Desired Outcome:

2. Key Participants:

3. Pre-read:

Yep, that’s it. Using this template and filling it in for a meeting you’re creating will instantly make it more accessible.

Other sites might give you anywhere from 5-20 bullet points to fill out, but after years of experimenting, these are the key three elements I’ve found most helpful. It solves the following questions:

• What are we here to do?

• Why am I here and why are others here?

• What do I need to know ahead of time to be effective?

Basic template, but gives attendees so much power to say “oh, I don’t need to be there.”

Desired Outcome

What are we doing here? The desired outcome instantly highlights the key purpose of the meeting. Some examples:

• Desired Outcome: Determine which front-end framework to leverage for XYZ project.

• Desired Outcome: Decide on where we will have our hot sauce festival

• Desired Outcome: Follow-ups and action items after brainstorming how best to cook a potato

Every meeting can usually be boiled down (potato pun) to one key sentence of why you are all gathered in a room, virtual or otherwise. When a desired outcome is noted, all attendees can help keep the meeting on track, because they know where it ought to be going.

Key Participants

Highlight who is in the meeting and what purpose they are going to serve. Continuing our potato cooking meeting, it might look something like this:

• Gary - Vegetable cooking expert

• Brenda - Viral cuisine consultant

• John - Potato enthusiast and part-time potato-eater, general food SME

Again, straightforward, but tells each person what part of their brain you want to tap into in the meeting. It also gives each individual a chance to tap someone else who may be a better fit for the meeting or the expertise you seek.

Expertise ought to vary if you’re making a decision. Otherwise, you run the risk of building an echo chamber. If your goal is to have everyone nod their heads and agree with whatever idea you came up with, don’t hold a meeting, simply say “this is the decision we’re making” and go from there.

Note: A general rule of thumb is to keep participants to 4 or fewer. There are exceptions, but challenge yourself to keep it at 4 or fewer and see what breaks the rule.

Pre-read

When there is a lot of data, complex ideas, diagrams, or anything that needs to be digested ahead of time, links to pre-reads are helpful. You don’t need to go to Amazon lengths of memos and spending the first fifteen minutes reading the memo, but help others be prepared to best help you.

Too many meeting attendees hop in and get blindsided with overly complex information that in won’t sink in until after the meeting, making it tough to make a decision and generally spawning at least one more meeting. Be thoughtful and think, “if I were attending my own meeting, what would I need to know to feel comfortable at the table.” Link those items in the Pre-read section.

Worst-case scenario? No one reads it. Best-case scenario? Everyone reads it and comes prepared and the meeting is done in six minutes. The prospect of the six-minute meeting wins out in my mind every single time.

Does it create more work for you as organizer/facilitator? Sure, but guess what, a side effect of you being more prepared means you will also be more effective.

Remember back in high school or college when your teachers would say “alright, you can have one note card of notes for this exam,” so you spent the night before writing as small as you could to cram everything onto one note card? Those teachers tricked you into better understanding the information. Walking in the next day I’d bet you didn’t use that notecard nearly as much as you expected. Solid trick.

Optional Means Optional

The heading says it all. If you’re adding someone for visibility or as an optional participant, mean it. Sometimes all but 2 individuals in my meetings will be optional, because I know who my key participants are and past that if folks want to be in there they can, otherwise I’ll send out notes/decisions after the meeting.

Especially with engineers, they may feel pressured to be in a meeting even if they don’t expect to be able to provide value. Adding them as optional and then reminding them what optional means, gives them the chance to opt-out.

Subscribe now

Meeting Length

Start with 25 minutes. As C. Northcote Parkinson so famously stated, “It is a commonplace observation that work expands so as to fill the time available for its completion” (The Economist, 1955). The whole essay is a solid read, but the point is still missed by many nearly 70 years later.

If your meeting is scheduled for an hour, chances are you’re going to take at least 40 minutes. If it’s 25, you might feel the time crunch, and become more focused.

Start chipping away at the knee-jerk reaction of scheduling for an hour. Surprise, you can always schedule more meetings. It’s not optimal, but it will put you in the mindset of “how do we work effectively for the next 25 minutes?”

Trust me, your teams will thank you.

Final Thoughts

This is by no means a perfect system, but it’s an easy system to implement and begin tweaking to fit your needs. If you are asking people to take time out of their day to help with something, it only seems fair to give them the tools to best help you.

What did I miss? Let me know below if you implement this template or another and the positive changes you see from it!

Quick Link: Competency Quadrant Example Spreadsheet [MAKE A COPY]

If you’re at a newer company, get tight on what the differences look like between levels and how expectations change at each level. It will save you heaps of hassle in the future.

The Core Competency Quadrants

The idea is simple, you’ve got four quadrants for every chunk of learning you do in life. At any moment, you can fall into one of these four quadrants.

1. Unconscious Incompetence: You don’t know what you don’t know

2. Conscious Incompetence: You know what you need to know, but you’re not any good at it yet (that’s okay!)

3. Conscious Competence: You know how to do the thing, but you need to think about it as you’re going through

4. Unconscious Competence: The skill becomes automatic and you don’t need to think about it

The standard progression is from Unconscious Incompetence or Conscious Competence into the other two. Generally, you learn about which areas you are unconsciously incompetent in through feedback from others, or reading books, blogs, and newsletters and realizing you’ve never heard of or tried something.

Think about your current role, where do you fall in each of the categories?

Note: Incompetent has a negative connotation, so be sure to keep in mind, and remind those you go through this exercise with, that it is acceptable to fall into the incompetence side of the quadrants, but once you’re there, the expectation is you will work to get to competence.

Fun Fact: I lost in the first round of the County Spelling Bee in grade school on the word “incompetent,” my parents couldn’t stop laughing at the irony.

The Competency Quadrants in Practice

Once you’ve learned this framework, you can apply it to pretty much any part of your life. Learning how to draw better? Use the quadrants to find your weak spots. Learning a new language? The quadrants can help you understand which areas to spend more time in.

Let’s put this into practice.

For this part of the example, we’ll use a job posting for a VP of Engineering position at Photobooth Supply Co. I’m currently a Director of Engineering, so I like to look at VP of Engineering and CTO roles (even when I’m not actually looking for a new role) to make sure I’m on the right path skill-wise.

VP of Engineering Responsibilities (abridged):

• Architecting, recruiting, and training a team to develop our product roadmap in 2023 and beyond

• Craft and constantly improve upon KPIs and OKRs to increase team output

• … a handful of other responsibilities …

• Manage software development projects by setting requirements, goals, and timelines

• Review and update policies relevant to internal systems and equipment

Assess Each Skill or Responsibility in a Vacuum

Don’t look at the whole picture yet, that’s not the point of the Competency Quadrants. We’re looking to take micro bites of a role and understand how we do in each section. For example:

Architecting, recruiting, and training a team to develop our product roadmap in 2023 and beyond

When assessing myself on this skill, I’d give myself a Consciously Competent. I’ve done all of these tasks in the past, many of them this year and last, but I still have to spend a lot of time on them. Reading it a different way, I’ve never trained a team to create a roadmap, so if that is the definition then I’m on the edge of Unconsciously Incompetent. Now that I know that is a potential skill gap, I could begin to work on it.

A few potential goals I could create off of this:

1. Create a framework for building out a reusable product roadmap and apply it to H2 2023 and H1 2024, evolving the framework with each use.

2. While creating the H2 2023 roadmap, collaborate and teach my managers how to effectively build a roadmap on their own

   1. Follow-up goal: Oversee the creation of the H1 2024 roadmap, coaching along the way.

Skipping down a few responsibilities, we get to this one:

Manage software development projects by setting requirements, goals and timelines

This falls into the Unconsciously Competent category for me. I’ve been executing projects with milestones and requirements since I was a freelancer in 2012, and have only gotten better since. I know how to break down a project into sizable chunks, assess dependencies, obstacles, and nice-to-haves, and can pretty much do it in my sleep. That level of competence is what you’re looking for in the Unconsciously Competent quadrant.

Those are only two quick examples, but it gives you a glimpse into how effective the Competency Quadrants can be for assessing skill level. I’m on the complete opposite ends of the spectrum for the two examples, and that’s totally fine! It means I have room to grow, and that is an exciting proposition.

Note: Going through this exercise is only effective if you are honest with yourself. You should discuss this with your manager once completed to validate the results.

When To Use The Competency Quadrants

I love using this framework for goals and career advancement discussions. It takes out a lot of subjectivity and bias and focuses on the role responsibilities within the next level of career advancement.

If you do not have a clear view of what role responsibilities look like on your team, work with your HR department to get them solidified. Another good place to start is to outline what your current expectations are for each individual on your team.

Core ways I’ve used the Competency Quadrants:

• Yearly performance reviews

• Goal-setting for myself and my direct reports

• Promotion or merit cycle letters of recommendation

• Role expectation clarifying conversations

• Performance management

I’m sure there are additional times when the Competency Quadrants are helpful, but that’s where I’ve found the most value so far.

For this next section, it’s important to remember that this information should never be used to force rank your individuals or compare them to one another. Each bullet point should be reviewed in a vacuum and each individual should be reviewed in a vacuum as well.

“Well Alan got an Unconsciously Competent on ‘Conducting Code Reviews’ and you got Unconsciously Incompetent.” If you’re looking to sow seeds of chaos, unhealthy competition, and distress, definitely compare folks against each other.

If you’re looking to run a healthy organization (like I hope all of you are), continue to remind folks that this is not for comparative purposes, but to see, based on expectations for them in their role, where they are in their own journey.

Taking Competency Quadrants One Step Further

For those of you that are data-minded, you can translate all quadrant values into numbers to get an average score and get “one number to rule them all.” The number should be used as a quick reference only, not as a direct measurement. Context is quickly lost in numbers alone.

I never have folks directly connect numbers to the responsibility in the same sheet, because subjectivity and bias start to creep back in as you realize you have mostly 0s and 1s.

Instead, have each responsibility as a line item, with a dropdown for the quadrant, and then a notes section. You can also add a “Manager Notes” section if you like.

If you create your own, it might look something like this (though you could just copy my spreadsheet here):

The Competency Dropdown makes it easy to see exactly where you feel you sit with each responsibility. It doesn’t matter if there are 3, 10, or 50, you can keep adding to that spreadsheet to keep track of how you’re progressing.

Once the Assessment tab is filled out, you can get a quick glance at your “score” on the Data tab of this spreadsheet.

With some spreadsheet magic, it will count up all the quadrants you or an individual fall into, then give a “score” between 0 and 3. You could then make the following assessment, though it will vary based on role and company:

• 0-1 - New to the role, or needs improvement quickly

• 1.01 - 2 - Definite room for growth, but solid overall

• 2.01 - 2.5 - Doing well, potentially time to look at a promotion in the next 6-12 months.

• 2.51 - 3 - HOW IS THIS EVEN POSSIBLE?!

I’m generalizing the above because only you will be able to tell the full story. The number alone should not be the lone indicator.

The real meat here is to look at the other tabs, which show a breakdown of each competency (UI/CI/CC/UC). Goal-setting usually will take place in the Unconsciously Incompetent and Consciously Incompetent tabs.

These pivot tables help break down each item that was marked as one of the quadrants to give a focused look at where improvements can and should be made.

Armed with this spreadsheet and a list of responsibilities for your role, goal-setting should now be a breeze!

Final Thoughts

An added benefit of breaking down responsibilities in this fashion is you can create a running log of activities that help carry you into the next quadrant of competency. That running log makes it extremely easy during the next promotion or merit cycle to show what was accomplished. It’s essentially a more detailed hype doc, with concrete facts and less hype.

I’ve been using the Competency Quadrants for 4+ years now and it continues to be helpful, especially in those tricky situations where goal-setting is difficult. Check on these at least twice a year to review progress and adjust ratings.

Let me know below how you end up implementing the Competency Quadrants or if you have any questions/feedback!

I’ve been conducting 1:1s for over seven years now and have iterated between large agendas, no agendas, and every variety in between. From all the years of working with individuals, I’m happiest with this agenda that I’ve put together over the last year. It’s pretty straightforward, but take a look and I’ll explain below.

Keep in mind; any agenda can work, the important part is to meet consistently. The (R) indicates recurring topics which can be added or removed as the individual sees fit.

Agenda

• (R) Personal check-in:

• (R) Process Feedback:

• (R) Team Feedback:

• (R) Goal 1:

• (R) Goal 2:

• (R) Goal 3:

• Topics to Discuss:

10 is best, 1 is worst
Stress (1–10) 
Goals (1–10) 
Alignment (1–10) 
Career Path (1–10)
Feedback (1–10)

Notes

That’s it, that’s the starting agenda.

Agenda

Pretty straightforward, the Agenda section has (R) recurring topics and then free-form topics, whatever the individual wants to add for the week. I like to add goals on here as a recurring topic to see how things are progressing and what I can do to assist, but if the individual finds it to be stressful to include those, take ’em off!

The substance within each bullet can be anywhere from one word, to an entire novel. Either way, discuss and determine any necessary action items or next steps.

There are two sections about feedback. This is intended to give chances for the individual to bring up things that may be going great and we should do more of, or things going poorly that we should fix or stop doing. If the individual consistently does not have any thoughts here (positive or otherwise), assign it as homework. If they still don’t have opinions there, that’s fine too.

Revisit your templated topics quarterly and see if there are recurring topics to add or remove.

Note: Some folks shy away from the “Personal Check-in” on the individual side and that’s okay. If someone is uncomfortable discussing anything outside of work, that is their choice and it should be respected. I’ve found that trust is more easily established when you understand more about the individual, but if they draw a line, absolutely do not cross it.

Data

Tracking data in a 1:1 may be a bit jarring for some folks, but it is not intended to be a super official number. The Data section is to give a quick gut check of how you’re feeling in certain areas, and is sometimes the only part that gets filled out.

The quick assessment gives a quick look at how everything in the world is affecting the individual, with each category digging into a different area.

Stress (yes, I know it seems backward): How’s your stress level this week? If it’s a 10 (no stress), how do we keep it there? If it’s a 1, what do we need to do immediately to get you into a better situation?

Goals: How do you feel you’re tracking with your goals? Lower numbers tend to point to a lack of clarity around goals or a lack of time able to be spent on those goals. Higher numbers mean we’re on track and feel good about knowing what to do. This should also reflect the updates in the recurring section about goals. Knowing growth is not linear, this one could fluctuate quite a bit.

Alignment: No, not Chaotic Neutral, or Lawful Good, Alignment focuses on how aligned the individual feels to company goals/vision, team goals, or their own goals. Alignment can make or break engagement, so I like to ensure the individual can address alignment each week. This is especially helpful during times of change.

Career Path: How do you feel about your career path here? Do you have one? Is it going as you expected? Lower scores here indicate you should spend some time determining the current role scope, desired role scope, and how that fits into the company at large.

Feedback: Are you getting enough feedback? From me? From your team? Enough constructive or positive feedback? “Not enough feedback” is a frequent complaint surfaced to managers, so I like to ensure we have a chance to address it weekly.

You can also ask the individual if there are any other categories they’d like to track. Look for anomalies and historical patterns. Do not, and I repeat, do not try to compare the numbers of one individual to another. That’s not the point of this exercise.

Notes

This section is for shared notes, or the individual’s notes during/after your conversation. Determine between the two of you what you want to use it for and go from there.

Note The Lack of Project Updates

Too many early managers make 1:1s about project updates (this is also a sign of a bad manager). You can check on projects elsewhere, so don’t clog your 1:1s with a checklist of projects being worked on.

If there are chunks you’re working through together, sure. If you want to review PRs during this time, that’s also fine, but make sure it is the choice of the individual. This is their time.

Don’t Force It

I do not require my direct reports to fill out every section. My standard ask is, if nothing else, please put in the data at the bottom. A gut check is enough to go off of, and from there, if there’s really nothing to talk about, that’s fine too!

How about you, how do you like to conduct your 1:1s? Any particular template that has been meaningful or worked well for you?

The Helm is always publishing great ideas, insights, and advice by leaders, for leaders. To make sure you’re always seeing articles like this one, subscribe to The Helm on Medium or check out our independent site for free & exclusive content.

If you have your own ideas to be featured in The Helm, drop us a line. We’d love to hear them.

Velocity can be a helpful numerical representation of work completed for executives and managers, but it doesn’t tell the full story. It leaves out critical information about what happened over the course of a sprint, regardless of the length.

Can velocity answer:

• How many folks were out sick or on PTO for that week?

• Cards that were added/removed?

• Points added/removed?

• General scope volatility trends?

• Other questions about sprint fluctuations?

The answer is no.

If you are only measuring a team by velocity, you’re going to get gray hair fast and your teams will begin to despise you. Below I’ll outline some additional metrics that I like to keep track of to understand the ebbs and flows of work on a team. This is not a stick to measure people with, it’s a tool to understand the volatility you and your team face from sprint to sprint.

For all of these metrics, volatility should minimize over time, making it easier to understand anomalies.

Here’s a Google Sheet with 2 sample teams.

Net Estimate Shift +/-

Depending on how your team approaches estimates and adjusts them in the sprint, Net Estimate Shift may be irrelevant.

When I’m starting with teams, we like to challenge estimates as we get into the work to see if we’ve learned anything new between estimation and starting the work.

Generally, this number should be low but add some conditional formatting to understand when estimations are shifting frequently.

Points Added / Removed

Especially if you are on a team that is actively supporting an application while also moving into the next phase of work, Points Added and Points Removed can give you a sense of what new work is coming into your sprints. Higher volatility here could mean the requirements were unclear, not thorough enough, or in some cases, a stakeholder changed their mind mid-sprint.

When Points Removed is high, analyze why that happened.

• Was there a chunk of work that was de-prioritized?

• Was something unclear at the beginning of the sprint that crystallized?

• How do you get ahead of that in the future?

Weighted Team Projection

Note: Weighted Team Projection is more for estimating your next sprint. You will need 5 sprints in order for Weighted Team Projection to start becoming useful and even then it will need several months before it starts lining up more accurately.

to

The equation:

.05(s1)+.1(s2)+.1(s3)+.25(s4)+.5(s5) = WTP

Weighted Team Projection looks at your last five sprints (s1-s5) and adds weight based on how far in the past the sprint was. The weights are added to each of those sprints accordingly (5%, 10%, 10%, 25%, 50%).

Once added up, Weighted Team Projection gives you a rough estimate of what your team should be able to accomplish in the next sprint. The projection is helpful when looking at a reasonable amount of work to complete for the next sprint.

Carryover ∆

Pretty simple, but this is a percentage of how many points are carrying over into the next sprint vs how many points were committed. It is a raw data point and doesn’t take into account why things changed.

Adjusted Percentage Commitment Met

Not everyone agrees with me on this one, but that’s alright. The intent of Adjusted Percentage Commitment Met is that it factors in everything that happened throughout the sprint and gives you an assessment of, all things considered, how did we do?

For me, a tolerable number here is between 80% and 120%. A great number is between 90% and 110%. Think of APCM (what a great acronym eh? /s) as the exhaust port on the Death Star. There’s a bit of wiggle room in either direction, but the further away from 100%, the less desirable the outcome.

Scope Volatility

Pretty straightforward here, but scope volatility takes a look at Net Estimate Shift, Points Added and Points Removed, divided by the total Committed points in your sprint.

Scope Volatility is our best percentage look at how things shift during the sprint and whether we are blowing the sprint up by adding or removing points, and whether we are adjusting estimates without balance. It’s definitely a quick reference number, similar to Adjusted Weight Completed.

Another Look

With the above in mind, how would you feel about your team’s velocity if it looked like this?

Your answer should now be “it depends,” but it definitely starts to represent what we want to see from the velocity graph. A positive trend, generally within 10–20% of initial commitment, with the variability calming down over time.

Free GSheet Template you can try out yourself!

Final Notes

All of these numbers are easy to find in Jira. You could automate everything through some fancy Google Sheet scripting and a connection to Jira Cloud for Sheets Add-On, but it took about 15 minutes every two weeks, so I never got around to it.

If you are unable to find these numbers, comment below and I’ll help you out. Most can be found in the Burndown chart and Sprint Report within Jira.

Hopefully, the above metrics will help shed a bit more light on the volatility of a sprint that pure velocity absolutely cannot measure. Amanda Quint wrote another great post called “3 Metrics for Engineering Team Success Other Than Velocity” which gives some other ideas as well.

In a vacuum you can make up any story you want, so look at the story told by all of your metrics before making process changes.

Above all else, think about whether the outcomes are captured in the numbers. If not, then figure out how else to capture the outcomes you’re looking for.