It’s critical that applicants have a clear understanding of what to expect in the assessment phase of hiring. This is an opportunity to reduce any unwritten rules and make sure everyone is equally aware of the “tricks of the trade” when it comes to both technical and non-technical assessment.
Use the application as a pre-assessment
One way to end up with a manageable pool of applicants to interview at the right readiness level is to use the application as a pre-assessment. From an equity lens, you’ll want to design the initial application in a way that lowers barriers to entry. Simultaneously, you’ll seek to target individuals at the right readiness level to avoid interviewing people who are too early (or too far along) in their technical skill development and less likely to succeed in the program.
Things to consider when equitably designing the application:
What readiness level are you targeting technically and non-technically?
- What are the top 3–5 specific baseline technical skills needed to be successful in the program?
- What are the top 3–5 specific, baseline “non-technical” attributes sought after in strong candidates?
- Are we including a very clear, specific definition of these skills and attributes in an application guide?
- Are these specific skills listed appropriate to the work an apprentice will be expected to contribute? (Remember, they won’t be a full-time employee yet and should not be expected to already contribute at the same level as an entry-level technical hire. If they are at that level, they should be applying to one of those roles!)
- For non-technical skills, are we prioritizing a thirst for learning, openness to feedback, a healthy learning trajectory, and potential to grow?
- For technical skills, are we making sure to not screen at too advanced a level, given candidates are applying because they need to learn more?
What will the initial application assessment elements be to help narrow down who to interview?
Examples include:
- 2–3 short essay questions to gauge growth-mindset and non-technical attributes
- A resume to understand learning trajectory, transferable skills
- A coding/portfolio example to get a sense of their current technical skill level
- A take-home exercise to gauge technical skills
- Technical reasoning questions to gauge thought process and understanding of technical trade-offs over implementation of a solution
- Have we clearly explained the expectations for each of these elements?
- Have we provided an example of a successful application component (coding sample, structure for short essay question)?
- Have we suggested free online resources to help get someone started prepping these elements to help level the playing field?
How much time do you want to allow applicants to complete the application?
- Consider that narrow time-limits can be stressful
- Also consider the risks of making your application accessible only to those with a lot of free time — for example, if administering a take-home exercise putting adequate guardrails in place around length of time ensure those with unlimited time are not at an advantage compared to others who may be balancing other work or family responsibilities
- At a minimum, two weeks is recommended to allow applicants to complete all elements of the application
Once you’ve answered these questions, ensure these details are included in your application guide.
Design an Equitable Technical & Non-Technical Interview
Apprenticeships provide an exciting opportunity to re-invent a more equitable and effective model for technical assessment.
The most commonly used format for technical interviews is notoriously inequitable and ineffective in that it:
- Test for the wrong skills, focusing on advanced knowledge of complex data structures and algorithms that are unlikely to be used by early-career talent in the actual job
- Reward knowing the “unwritten tricks” of interviewing, much like standardized tests, which have been shown to disadvantage underrepresented people of color
- Have spurred an industry of expensive technical interview prep-courses needed to “beat the test” that are often cost-prohibitive to many communities of color
- Are often set up to intimidate through strict time pressure, austere interviewers, and pass-fail formats
Through the above, many qualified Black, Latinx, and Indigenous candidates from alternative or under-resourced education backgrounds are screened out. Thankfully, there’s a growing movement in the tech apprenticeship space to improve these assessments and also help these innovations spread to hiring for other technical roles across the company.
Here are some key design tips and questions to ask for the technical and non-technical interview:
Start by identifying the appropriate technical and non-technical skills a candidate will need in order to convert
Questions to Ask:
- What are we actually asking them to do in the entry-level role they will ideally convert into?
- Does the current career framework make space for people to enter the company from an apprenticeship?
- Do new career bands need to be implemented for the apprenticeship to feed into?
Pare down this set of skill to those a candidate needs to have prior to joining
Questions to Ask:
- Of the skills outlined in step 1, which skills will they be learning and developing during their apprenticeship? Which should they already be familiar with?
- What degree of familiarity and exposure will be expected prior to joining the program?
Create an assessment rubric that aligns with the baseline skills needed
Question to Ask:
- How can we test for only the baseline skills needed? Remember, we’re not testing for the full set of skills they’ll need upon conversion. They’ll have time to learn and grow those skills
- How can we maintain focus only on those baseline skills without baking in underlying assumptions of other (unstated) skills as well?
Tips:
- Create an interview evaluation rubric tracking the technical and non-technical baseline skills and core values sought after, with concrete examples of what performance looks like in terms of exceeding expectations, meeting them, or needing growth
Create a technical assessment/interview structure that tests the baseline competencies you’ve prioritized in a way that builds confidence
Question to Ask:
- How might we craft an interview experience that doesn’t overwhelm or intimidate?
- How might we provide candidates with a positive interviewing experience, even in the case that they don’t get as far along as other candidates?
- Do all interview questions/processes map over clearly to our baseline competencies?
- Have we avoided testing unnecessarily advanced skills such as complex algorithms/data structured (or equivalent in UX, product, etc)?
Tips:
- Create a series of smaller challenges that test identified key competencies so it doesn’t feel like a pass-fail interview
- Assure candidates that this is a learning and growth opportunity
- Structure the interview as a pair-programming session in which learning and problem solving is the objective
Use the technical interview to assess non-technical skills
Mastery of non-technical skills is almost a better indicator of success in apprenticeship programs than technical skill mastery
Question to Ask:
- How might we track non-technical performance in our technical interview?
- How might we use the technical interview as an opportunity to gauge growth-mindset, coachability, or willingness to ask questions?
Tips:
- If interview is conducted live, explicitly state that candidates can think of the interview as a pair programming session and encourage them to ask questions/ask for help as they problem solve with their interviewer
- Track indicators of any non-technical skills you’ve identified as necessary baseline skills — for example, growth mindset, how they communicate challenges, whether candidates are asking questions proactively, or whether they incorporate feedback from their paired interviewer into their next steps.
Align the non-technical interview with the core values/ethos of the company and assess for growth mindset
Question to Ask:
- Which of my company’s core values are we seeking candidates to align with?
- If our company doesn’t have core values, how might we structure an appropriate value-set/ethos for this apprenticeship program?
- How might we structure the non-technical interview to assess for alignment with this ethos/values and evaluate for mindset, motivation, aptitude for growth?
Tips:
- Ensure candidates and interviewers know which clearly defined core values are important to align to ahead of time
- Explain to candidates and interviewers ahead of time how to use the STAR situational interviewing method. This method highlights a candidate’s behavioral actions in a specific situation they’ve encountered and the impact they had.
- Explain ahead of time to your candidates and interviewers the value of highlighting/recognizing transferable skills from the candidates’ prior experiences in seemingly unrelated roles or life experiences
Recruit interviewers who have been trained to work with apprenticeship candidates and assess for growth potential
Question to Ask:
- How might we ensure our interviewers understand the mission of the program and the alternative backgrounds of candidates?
- How might we prepare interviewers to assess based on growth potential?
Tips:
- Structure a training or briefing of technical interviewers to prepare them
- Ideally, interviewers opt-in voluntarily vs being forced to participate, which results in more mission-alignment with the program
- Avoid tapping interviewers who are only accustomed to senior level interviews
- Tap interviewers who come from alternative backgrounds themselves or have previously interacted with the apprenticeship program
INNOVATIONS TO LEARN FROM: To learn more about equitable innovations in technical interviewing, check out Byteboard, a start-up helping companies redesign technical interviews so that candidates “prep for the job, not the interview.”
— — — — — — — — — — — — —
Tool and best practices co-designed with champion in tech Jacqui Watts.
Have questions or comments about the Equitable Tech Apprenticeship Toolkit? Send us a note.
