Separating Hype and Reality
In the ever-evolving world of technology and innovation, there's a perennial tug-of-war between hype and reality. Just look at quantum computing as an example. It's the blockbuster movie of the tech scene, full of vision, promise, and potential. But here's the kicker: while it promises to revolutionize computing, it's been a very, very, very tough nut to crack. For all the headlines and hype, those in the know chuckle that quantum computing has been "just 10 years away" for the last half-century. Some things, it seems, are always just around the corner.
Singing a similar song, the HR scene over the past year has been humming with talk of skills-based, well, everything. “Skills” is the latest hit single on HR radio, playing simultaneously across recruitment, training, and career pathing stations. And with every new tech tool popping up, with front row seats to the latest skills show, the hype is certainly real and expanding. But for me, here's the million-dollar question: When the talk turns to paying for skills, are we getting ahead of ourselves? It feels a bit like waiting for that quantum computer to finally hit the shelves.
Switching hats between an economist and a data scientist focused on skills, I've got to spill the beans: the dream is way ahead of the technical reality right now. Over countless coffee chats this past year, one thing's clear: if someone's pitching you a skills-based pay solution, you might just be looking at a dressed-up old PC pretending it's the next-gen quantum whiz machine.
In a new series of blog posts, I will cover topics in the domain of skills-based pay and highlight the current technical challenges. My goal is to provide a perspective as a data scientist and economist on why we should be cautiously and judiciously reviewing skills-based pay solutions in the near term. Otherwise, we may just end up in scenario where skills-based pay is perpetually 10 years away.
In no particular order, here are some of the topics I’ll be covering in the series:
Skill interdependencies: A typical job requires a combination of diverse skillsets to perform value-added activities. The marginal value of skill is likely to depend on the presence of other skillsets. To measure skill premiums accurately, we must develop models that account for high-dimensional skill interdependencies.
Skills-based Pay is a Causal Inference and Prediction Problem: Skills-based pay requires us to attribute causal relationships between skills and pay. That is, if a worker learns a new skill, how does this new know-how affect pay? At the same time, our models must be capable of producing precise estimates for the magnitude of a skill pay premium. High variance estimates will not be useful.
Accounting for supply and demand-side factors: Skill premiums are likely influenced by the outcome of a matching process between employers and job seekers. Without comprehensive data on both demand and supply of skills, it will be challenging to account for how market dynamics affect skill premiums.
Unobserved proficiency levels: Skills-based pay extends beyond a binary approach that considers only whether an employee is competent in a given skill. Instead, skill premiums will also account for how proficient a worker is in a given skillset. Without sufficient data on these proficiencies, skill premium estimates may suffer bias.
Knowledge capital depreciation: Skills are fundamental elements of the human capital stock in an organization. As a knowledge stock, skills can atrophy, be obsolesced, or otherwise depreciate in value over time. From this view, model frameworks must account for how skills depreciate to avoid wage stickiness along the skills dimension.
The series will roll out over the next several weeks, and I may come back to the series whenever new ideas come online. Today, I’ll tackle some of the technical considerations regarding skill interdependencies.
Skills, They’re Better Together
You know how some things just work better together? Coffee needs that splash of cream, peanut butter pairs with jelly, and pretzels? Perfect with a cold beer. In the job world, it's the same tune. A role doesn’t thrive on just one skill but a medley of them. The value an employee brings is like a band playing in harmony, not a solo act. So, when someone says they can pin a price on a single skill, remember: it's like trying to judge a song by one note. Valuing a lone skill without considering its partners? That misses the beat.
Here’s the breakdown:
Omitted Skill Bias
When we try to understand the relationship between skills and pay, the interplay between different skills adds layers of complexity. Let’s consider a basic linear regression model where we try to predict an employee’s pay differential based on whether they possess a particular skill:
Now, for this model to give us an accurate representation of the pay premium attached to Skill #1, there’s a crucial assumption required: the error term (or the difference between predicted and actual pay) should not correlate with the presence (or absence) of Skill #1. Mathematically speaking:
But, here’s the catch: If we overlook other essential skills that (1) influence the pay differential and (2) are correlated with having Skill #1, then the crucial assumption is wrong. As a consequence, we will either undervalue or overvalue the skill’s contribution to pay differentials among workers.
Often, this assumption is violated because one skill along doesn’t paint the full picture in terms of job performance. As most musicians need both hands to play a song, many roles require multiple skills to create value in the organization.
But, even after accounting for all skills, we may still end up with biased estimates for the contribution of a skill to pay. To see why this might happen, let’s keep things straightforward and assume an employee could have two main skills, Skill #1 and Skill #2. Assuming these two skills don’t come with any hidden baggage (like being linked to other unobserved factors), our model might look like this:
If our assumption holds, the estimated pay premiums for each skill should tell us how much each skill contributes to pay.
Complimentarity Bias
But hold on, there’s another twist. Let’s say we find that Skill #1 contributes a certain amount to pay. Because of the model we specified, the value we estimate explicitly assumes that Skill #1’s contribution to pay is independent of an employee having Skill #2 too. That’s like having a nice glass of Cabernet Sauvignon without pairing it with a slice of dark chocolate cake. The wine’s flavor is enhanced by the chocolate. In my own role as a Data Scientist, knowing Python has value, but its real value shines when combined with machine learning know-how.
To address this modeling assumption, we can tweak the model’s specification to account for complementarities between skills:
In this setup, the contribution of Skill #1 to pay also factors in the precense of Skill #2. To put it formally,
What’s on the roadmap?
As demonstrated above, dealing with skill complementarity can be a challenging task from a modeling point of view. Methods from deep learning offer promise since neural networks are designed to handle complex interactions in high-dimensional feature spaces. However, there’s a caveat: these networks can be enigmatic, often obscuring the underlying mechanics of their decision-making. In other words, model interpretability will suffer, and pay transparency is all but out of the window. And, while neural network precision can be remarkable, these methods are prone to overfitting on training data, potentially compromising adaptability. It’s ultimately a delicate balance, ensuring predictive power and adequately representing complexity without sacrificing transparency.