Talent blackhole: Big data specialists thin on ground

While companies seek to leverage big data against their competitors, the main struggle is a talent shortage, new research indicates.

BY Cameron Edmond 14 Aug 2013

By 2015, 4.4 million IT jobs will be created to support big data globally, but only one-third will be filled, according to a report from Hudson released today.

Hudson’s research, detailed in Hudson Industry Leaders Series: Tackling the Big Data Talent Challenge, revealed that 78% of Australian respondents feel their business is not currently equipped to undertake a big data initiative, as they lack the relevant skills and competencies.

“Much has been written about big data and organisations know that there is great value to be leveraged from the volume, variety and velocity of modern data sources,” Mark Steyn, CEO of Hudson Asia Pacific, said.

Steyn stated that few organisations are approaching the challenge of how to best resource the big data opportunity. “How do companies identify and recruit the right people, who will effectively manage, manipulate and most importantly, exploit these fast-growing data reserves?”

As big data is still – for the most part – uncharted territory, it is important to find all-rounders. Individuals who can blend a deep technical understanding with business and analytical skills, as well as customer and market expertise can help to form a bridge between big data and the rest of the organisation.

“Unfortunately these individuals are in short supply. This presages a skills crisis of vast proportions and is forcing organisations to look outside the usual supply network for talent,” Steyn said.

The research from Hudson found that the most important big data team member is the big data analyst. Like their counterparts, the analyst must have a blend of both technical and soft abilities. While this may seem daunting, it does indicate that the analyst role could be filled by those with a number of backgrounds, ranging from sales, IT, marketing and engineering.

Key HR takeaways

Forging an effective big data team is all important. As the field may remain untouched by many organisations, Hudson indicated the key functions required for effective big data management:

Program leadership. An individual with the vision to drive change, lead programs, ensure value creation, and interface with management. They may become the face of your big data team.
Data management. Data managers must have Operational ETL (extract, transform, load) capabilities, operational reporting, and be able to develop and maintain predictive models.
Domain expertise. They must have business level experience and acumen, all the while ensuring the relevance of insight.
Big data analysis. The priority. Must respond to business queries, produce accessible insights, standardise data and encourage automation.
Big data project management. Like any project manager, they must drive economics of scale and standardise analytics.
Campaign management. An individual who can liaise between IT and business departments – translating insights.
Campaign assessment. Must address value capture, value measurement and testing.

“Big data is an equal opportunity disruption because all enterprises have access to vast streams of data. Those that invest in exploiting the data, and take the time to carefully find, hire and retain the right skills mix will achieve competitive advantage,” Steyn said.