How can HR effectively measure employees' use of new tech?
In March 2025, Shopify CEO Tobi Lütke released a memo on X announcing that employees must now prove AI can’t do a job before asking for additional resources. The company is integrating AI usage into performance and peer reviews, the memo said, and exploring its use heavily during the prototyping phase of projects.
For HR professionals across Canada, this directive suggests a new trend: AI is not an optional productivity tool, but a core job competency that must be measured and nurtured.
It also raises questions: how is performance measured when the tools employees are using are changing every day? It’s a complicated question that is the research focus of Dilan Eren, assistant professor of strategy at Ivey Business School.
HR leaders can begin planning for employee AI assessment by rethinking productivity benchmarks, Eren says.
An obvious first approach is to start with output-based measurements.
“One measurement organizations will have will be related to speed, and looking at the numbers of tasks an employee or team complete in a given time,” she says.
“This really speaks to the core promise, core appeal of AI, right? This increase of productivity and efficiency.”
However, Eren warns that focusing too heavily on speed of task completion risks losing sight of more complex implications.
— tobi lutke (@tobi) April 7, 2025
Beyond the obvious benefits of faster tasks and efficiency, Eren outlines how skill “stretching” is a particularly exciting application of AI tools currently – and another area where performance metrics could come into play.
“It is enabling us to complete tasks outside our expertise area,” Eren explains.
“Without hiring a new expert, maybe we can actually use our current employees, equipping them with the right AI tools so they can again stretch their expertise, maybe a little bit, and be able to complete tasks without the help of an expert.”
This has special relevance for client-facing sectors – such as tech platforms like Shopify – where teams must adapt quickly to new project needs. AI can act as a force multiplier of sorts, by helping current employees meet varied client expectations without the need for hiring additional specialists.
For HR leaders, this suggests the need for skills assessments that reflect not only proficiency but flexibility – basically, the ability for an employee to expand their capabilities through the use of AI.
“Basically you can assess, you can look at an employee, ‘Okay, so far, she's able to do task A, B, C. But with that, now she's doing D and E … this could be another way of, again, measuring the impact of AI usage.”
However, she cautions against taking that task stretching too far.
“This has some potential pitfalls, and we really need to be careful about which tasks are done by AI, which tasks are done by the quote, unquote ‘expert,’ or the human,” she says.
“There will be some issues here in terms of quality versus quantity. If the stretch is actually just a little bit, it might work, right? But if I start to actually try to do something way further away … because it is so out of my expertise, I’m not actually able to judge the quality of it.”
In other words, the farther an employee “stretches” their job duties away from their regular role and into areas where they don’t have expertise or even familiarity, they will be less likely to spot errors issues of quality.
“So the more we try to actually use AI to stretch the expertise and without hiring other experts, the [wider] the gap … I strongly suggest that the level of stretch should be actually on the shorter side, because the more gap we have, the more it will be difficult for us to actually oversee the process and understand if this is actually good or bad.”
Another key theme from both Eren and the Shopify memo is that employers should think beyond outcomes. Instead, they should consider how well employees are integrating AI into their work through exploration and adaptation.
“We are focusing on approaching this issue of AI and skills in terms of the output... But I think the second approach will be focusing on the process itself,” Eren says.
“This requires some concrete steps at the organizational level, for enabling and creating these, maybe collective spaces for employees to experiment and again, explore, but without the pressure of output.”
Shopify’s GSD (Get Shit Done) project expectations mirror this emphasis on experimentation. By encouraging AI use during the prototyping phase, Shopify says it fosters a culture where innovation and trial-and-error are valued.
Eren suggests that HR leaders might consider structured AI sandbox sessions, or allocating work hours for experimentation: “Setting aside a few hours of work time – not after work, right within work time — for collective experimentation, and not rewarding who came up with the best idea, but just rewarding participation here, and celebrating failed attempts.”
Scaling AI adoption isn't just a logistical challenge, Eren points out — it's an emotional and cultural adjustment for employees. She highlights the importance of tone and transparency in leadership communication during the process of implementation.
“We need to acknowledge the fact that most of them, there is reluctance, resistance. They are afraid of some stuff, like automation, am I going to lose my job?” says Eren.
“Whether or not they are warranted, these are important concerns, right? I think it is more important for us to better communicate why we are having this need for AI, assuring employees that, ‘This is not to replace you. Actually, this is just another tool. We are trying to figure out the best ways of using it.’”
Eren points out a hidden challenge that HR professionals should anticipate: generational conflict arising from junior and senior employees adopting AI tools at different paces, whether due to aptitude or simply the time or capacity to spend on experimenting.
She points out that the introduction of AI tools into regular workplaces may be disrupting the status quo of expertise or experience-based hierarchies.
“Now we are seeing, maybe it is more likely for senior employees to be a little bit more resistant, or maybe not equipped to use technological tools, versus a junior employee with no expertise, but they have this aptitude for AI,” Eren says, adding that this is one point where HR can come into the picture.
“For HR, actually for the organizational health, per se, it is really important keep an eye on that too, managing this relationship.”
Drawing on both the Shopify AI policy and Eren’s analysis, HR professionals can begin implementing AI skills assessments and strategies by:
AI adoption could also impact traditional mentorship structures; in organizations like Shopify, where AI use is expected across roles, HR teams may need to intentionally design opportunities for collaborative learning.
Without this, Eren points out, AI adoption may lead to knowledge silos rather than knowledge sharing.
“Now we learn by watching others, more expert, experienced employees, right?” says Eren.
“But with the AI usage, workers are more isolated, and especially junior ones, are less and less able to actually observe how things get done.”
