If you live in today’s world, read the papers, listen to the radio, or (more to the point) get on the Internet, you’ve heard the phrase: BIG DATA. And maybe you’ve heard of BIG LEARNING DATA too.
We know big data is about data, and we know if we consult our friends at Merriam-Webster, they’ll tell us that in general terms, data is “facts or information used usually to calculate, analyze, or plan something,” and in terms that are more specifically relevant to this article, data is “information that is produced or stored by a computer” (bonus points if you happened to know that “data” is the plural version of “datum,” grammar junkie).
And of course, we know that the word “big” placed before “data” means there’s a LOT of data. It doesn’t really matter exactly how much data you’re talking about. It’s enough data that it’s hard to manage, analyze, and make sense of with common software applications (read: Excel spreadsheets).
But how much do you know about big data? And in particular, how much do you know about how it’s being used and will be used at your workplace, and how it will be used in your training programs and your learning & development programs?
If you’re a little fuzzy on all of this yourself, take a few minutes to read this article. It may provide a few “a-ha” moments, give you an insight or two, and help you better prepare for the big data revolution we’re told is coming soon.
We’d also very much value your own insights, thoughts, predictions, opinions, and comments in the comments field at the end of this article.
The term big data can sound intimidating. Who knows what it means? Who knows what it is?
But really, it’s all around you, you’re probably already familiar with it, and the ideas aren’t all that complicated, even if the methods require some significant know-how.
Let’s take a quick look. Remember, we’re not super-interested in discussions of just how big the data set has to be in order to be considered BIG data. It’s big enough to be hard to analyze, and it’s often coming from multiple sources. If you know that, you know enough for our purposes.
Big Data in Society
Big Data is used in our society all the time.
For example, retail stores accumulate data on their shoppers and their buying habits every day. They then pour over that data to find strategic advantages. For example, a a clothing store chain may use this kind of big data to compare the performance of different franchises in cities throughout the US.
In a similar way, people in the medical and healthcare industries collect and analyze patient data for diagnostic purposes, to set health care premiums, and more.
Big Data and the Internet
Google, Facebook, Amazon, and other Internet giants all use big data. In many cases, they’re doing it for advertising and marketing purposes, but this data can be used in other ways, too (we’re looking at you, NSA).
Companies have been collecting about their manufacturing processes for a long time. If you have a historian or a software application to help you visualize that manufacturing data to see trends, you already know that.
In the future, this is going to be even more true. If you’ve been reading about the Internet of Things, the Industrial Internet of Things and/or Industry 4.0, you know that too. If you haven’t heard about these things, you will soon–trust us.
Big Data in EHS/Safety
Safety professionals have always used data. And now, they’re collecting increasing amounts of data and putting that data to ever-new uses.
And that use of data in training is only going to increase, because people want to make sense of it to create better learning experiences for workers, to better evaluate materials, to justify the expenses and efforts of L&D, and so on. Just think of the increased creation of learning data from more scenario-based learning, more informal learning/social learning, more performance support, xAPI, LRSs, and attempts to more directly map training to performance.
Which is what we’re here to talk about today.
The Future of Big Learning Data: Sources and Uses
Although big learning data is still quite new and we can’t accurately predict everything it will eventually entail, there are already some clear indications of a few major trends.
We’ll introduce each below, starting with some sources of big learning data and the looking at some uses of that data.
Sources of Big Learning Data
You’re still going to be able to use the data you’re collecting on your training program now (which could arguably be considered big data already). This includes the stuff you’re probably familiar with already, and that your LMS is good at collecting–pass/fail data, completion/incompletion data, test scores, answers to multiple-choice questions, the time it takes to complete an activity, activities completed, and more.
A learning management system is specifically designed to record large amounts of workforce training data.
In addition, though, there will be new sources of training data, which we’ll introduce below.
Interactive Online Learning
Interactive online learning exists today, but it will be even more common in the future.
This includes:
Non-linear e-learning courses with “branching” options
Scenario-based online learning that puts the learner in situations like those they’d face at work and asks them to problem solve or make appropriate decisions/perform appropriate actions
Game-based learning in which workers learn or practice work skills within the context of a learning game
Virtual-reality simulations that duplicate the work environment and require workers perform simulated versions of work tasks
Here’s an example of one, below, although of course these will take many forms.
An interactive simulation of a web tensioning system on a paper machine can provide data about how each individual interacts with the simulation
These types of training are valued because they are highly effective. Why? For one, they’re highly engaging, so they capture the worker’s interest. Plus the provide practice on the real-life tasks the worker will have to perform on the job.
Not only will these type of learning activities become more popular, but they also provide more data about the worker. At each point in the inaction/scenario/game/simulation, data about what the learner is doing can be captured–what they did; how long they took; and so on. That will mean we’re collecting more data than the standard pass/fail, complete/incomplete, test answer, time to completion data we often focus on now.
Informal Learning
Informal learning is the stuff people learn on the job outside of the formal learning that’s assigned to them by their training managers. This includes stuff they learn directly on the job, often from their bosses and/or a mentor, plus stuff they learn from their peers, perhaps during water cooler chit-chat.
Nobody really knows how much people learn through informal v. formal learning. You’ll often see the 70/20/10 ratio thrown around (70% on the job, 20% from peers, and only 10% from formal training), but there seems to be a lack of data to support the exact numbers.
Nonetheless, nobody argues that informal learning does contribute a lot to workforce learning. And as a result, learning & development experts are looking into ways to facilitate, track, and make sense of that informal learning.
To that end, you’ll see more emphasis on things like:
You’re also going to want to draw data from performance support you’re creating, in addition to just training materials. What performance support is being used? How much? By whom?
Uses for Big Learning Data
So if we’re going to be collecting more data through our workforce learning & development programs, how will we use that data?
Again, there are many answers, but let’s focus on two big categories below.
Data Analytics and Training
Data analytics is the process of taking lots of data from multiple data sets, often from different sources, and analyzing them to find patterns, trends, correlations, and similar relationships that might not otherwise be apparent.
Data analytics can help us provide meaningful data to learners/workers, evaluate and revise our training materials, and more. And as the field of big learning data grows, we’ll be able to analyze that data in increasingly sophisticated manners, using it to find relationships that provide business value but are not immediately apparent.
For example, why did that forklift operator run into that wall? Can it be traced back to something from her training–and is it something you can fix for other operators? And why is your best sales person so good? Is there a specific connection with the training program here, too–maybe one that can be repeated with others?
Those are simple examples, but they’re the kind of thing you might be able to determine by analyzing big learning data more closely. And that’s the kind of thing you’ll be able to learn by comparing one set of data with another set of data drawn from your training program, or by comparing data sets from training with other sources of data, such as systems that track individual worker performance, your ERP, plant manufacturing data, and so on. The ultimate goal will be to better use your training program to create desirable business outcomes.
For an easy example, you can use data analytics to find what the training of your highest-performing sales people has in common, and then make sure that all sales people receive those same training benefits.
Want an example from a different context? Are you familiar with Moneyball: The Art of Winning an Unfair Game, the book (and later movie) about Billy Beane’s use of advanced baseball statistics to put together a championship-caliber baseball team on a small budget by assembling a team full of players who performed well in statistics that were not (at that time) valued as highly as more “glamorous” statistics such as batting average, home runs, and runs-batted-in? If not, check it out. It’s a good read and will help you appreciate the idea of finding relationships in data that aren’t immediately apparent.
Go to Amazon, buy a few books or view a few movies, and they begin to clue in on what your interests are and to automatically suggest similar titles.
Why shouldn’t L&D use the same type of suggestions to help workers find materials they’ll find helpful?
Evaluation of Elements Within Learning Activities
Big learning data tools will let you know how specific parts of learning activities are doing. For example, you’ve created an eLearning course and added an attached document with a “Read for More Information” link. But does anyone even click to read it? Big data will make it easier for you to learn about this.
ROI of Learning Efforts
Developing an ROI for L&D is a bit of a holiday grail.
Acquiring more data about the learners, the activities they complete (and don’t), and their performance at work will make it easier to evaluate an ROI.
Predictive Analytics and Training
If data analytics will help you discover relationships with your training data, predictive analytics will help you use those relationship to make predictions that can be helpful in a business context.
For example, maybe it’s something as simple as creating a prediction that people who score well on one particular activity during their onboarding program tend to be effective managers. And maybe you can use predictive information to begin grooming your next wave of managers. Or, maybe people who go through an interactive training course about forklift safety very quickly on their first time tend to have a higher safety incident rate with forklifts in the field. And maybe you’ll steer these people away from being forklift operators (pun intended!).
Lessons Learned from Reading “Big Learning Data,” a Book Edited by Elliott Masie and Published by the Association for Talent Development (ATD)
We recently read Big Learning Data, an ATD-published book edited by Elliott Massie that looks at big learning data and data analytics from many different perspectives. It’s short on answers and comprehensive how-to guides, because this is a new field. But it poses many helpful questions and provides lots of well-placed nudges in the right directions.
Sets the scene nicely, including some impressive stats on the amount of data we generate and the current (and projected future size) of the big data market.
But maybe most interestingly, it offers a three-pointed definition of big data, saying the “terms generally describes three aspects of data:” volume (meaning, there’s a lot of it); velocity (meaning, you can get it real-time or close to real-time), and variety (meaning, it comes from multiple sources). See page 2.
On Big Learning Data: Thoughts from Elliott Masie (Elliott Masie)
Touches on many interesting topics in a superficial, conversation-starting manner, and wraps up with a nice list of three major challenges facing us in the future:
How ready are we to manage, analyze, and use this increased amount of data effectively?
How ready are our learning systems to work with big data (not just in training/learning, but in all aspects of workplace performance/measurement)?
How can we distinguish between worthwhile, meaningful data and meaningless, “silly” data?
Why Are Big Learning Data and Data Analytics Important? (Nigel Paine)
Sketches out some reasons to use big learning data. This can be boiled down to two points:
Companies that make effective investments in learning & development (L&D) outperform other companies
You’ve got to use learning data to determine what’s an effective investment in L&D
This essay includes a nice 8-point list of things to consider when moving forward (with the main thrusts being: you can do it, get started now, and use data to tell a story), and ends with a note about preparedness to use this new data in new ways:
“The October 2012 Harvard Business Review ended one of its big data articles with the statement “Big data is an epic wave gathering now, starting to crest. If you want to catch it, you need people who can surf” (Davenport and Patil, 2012). This is true, but you also need people who can act on what the data is telling them, and you need people to be data-driven.” See page 22.
The Skills and Mindset Required for Big Learning Data (Donald H. Taylor)
Presents some solid points:
Special skills are necessary for big learning data analysis
Most learning & development experts don’t have those skills (I imagine that’s true of many companies as well)
One of those skills is knowing how to find and query data (this can be learned–“running a database query, searching across tables of data to find what you want, and running analysis using a tool such as Excel pivot tables to extract the information you need). See pp. 24-25.
The second skill, data analysis, is trickier
And, noting the difficulty of analyzing data, making accurate connections, and putting that information to use effectively, ends with a good cautionary note:
“The risk of big data is that rather than putting it to good use because we know what we are doing with it, we become the servants of those who do.” To that note, I’d add another risk is wrongly applying it when we only think we know what we’re doing.
Big Learning Data and Training Programs: Start Small to Go Big and Go Big to Go Long (Tom King)
Gives some examples of how you can use big learning data for new courses in development and older, established courses possibly in need or evaluation or revision.
Some interesting points raised concern:
The value of actionable metrics v. vanity metrics (an actionable metric is a metric that would result in an action and a vanity metric is meaningless information that may seem important even though it’s not–referred to as “silly data” elsewhere in this book)
The value of split testing (also called A/B testing) during course design to experiment with appeal, retention, or application)
The chance that our oldest courses, perhaps most in need of “big data help,” have little data behind them due to LMS migrations over time, etc.
The importance of drawing data from many sources, including non-training sources
The importance of patience (it takes a long time to collect large data sets that reveal meaningful information), persistence (keeping all historical data, and not overwriting/losing it when changes occur), and consistency (recording data consistently in the same formats over time and throughout multiple systems–think of something as simple as the many different ways in which dates can be recorded)
Three Roles of the Learning Leader (Coley O’Brien)
Lists three “roles” the author thinks are important for big learning data and analysis to work in an organization:
Connector–“someone who knows which data are available in organizations, has some basic understanding of what data tell us, and can provide recommendation as to how to best use data to solve real business problems.
Catalyst–“someone who knows which groups or individuals control the data and knows how to break through bureaucracies to help others get access to the right data
Content Expert–“Someone who has broad knowledge of business metrics and can provide guidance to analysts on key questions,” including (desired targets/acceptable ranges/data capturing frequency/importance of data sets/who uses this data and how/how is this data most easily interpreted by others?)
Stakeholder Perspectives and Needs for Big Learning Data (Rahul Varma, Dan Bielenberg, and Dana Alan Koch)
Discusses the importance of identifying and including different big learning data stakeholders at work and determining what needs they have from big learning data.
Also includes an emphasis on the importance of big learning data for improving training programs, skill development, and business results (pretty closely related to Kirkpatrick’s levels 1/2, 3, and 4, accordingly).
Avoiding the Dangers of Big Learning Data (A. D. Detrick)
Offers “a list of the four most common myths and misconceptions of what a big data environment can and should do”:
Deferring completely to digital data in decision making
Using data to confirm assumptions
Sharing too much data with employees
Expecting to uncovered predictive analytics quickly or consistently
Big Learning Data Risks: Privacy, Transparency, Culture, and Silly Data (Elliott Masie)
Notes that there are and will continue to be concerns about privacy, transparency, cultural issues, and meaningless/”silly” data, and proposes some common-sense starting points for those issues:
Transparency: “Learners have the right to know how learning data will be used, shared, stored, or leveraged.”
Privacy: “Organizations may want to define area where the privacy levels are different, or even whether the learner gets to indicate the desired degree of privacy.”
Value to the learner: “We must make big learning data valuable to the learner–or it will be a one-way and low-trust process.”
Silly data: [Data] “must have context, trust, and reliability to be effective…We should develop a set of queries that helps us evaluate the meaningfulness of the evidence and conclusions…How do we add context and validity to our big learning data efforts?”
Skill-building and collaborative examples: “Let’s define the skills, competencies, and approaches that managers, learning producers, and learners need to leverage big learning data”…and “build multi-organization collaborative efforts to provide tools, technologies, and analysis models that will push our big learning data competencies forward.”
Case Study: It’s Bigger Than Big Data–Metrics and Measurements Without a Strategy is Just Data (Nickole Hansen, Peggy Parskey, and Jennifer O’Brien)
Starting with the assumption that it’s important to have a strategy for using big learning data, this chapter is a case study of how one company developed theirs, and it offers a template for creating your own strategy as well.
The chapter ends with three additional takeaway:
Get leadership involved early
Look holistically at all the dimensions that affect measurement
Begin small and build from there
Case Study: Big Data, Big Difference in Training and Development (Jeff Losey)
This case study focuses on how a company used big data to create an online portal “to help our employees assess how they measure up against the 47 skills, 14 competencies, and five major themes we’ve identified as mission critical” to our company. In addition, the portal “enables every employee to design, track, discuss, and share their performance and personal growth with others in the organization.”
Case Study: Points for the Training Scoreboard (Ben Morrison)
A case study of a company trying to come to terms with, and create a dashboard for, evaluating training effectiveness along the common four-five level evaluation model:
Learner reaction
Learner knowledge
Job transfer
Business effects
ROI
Case Study: Big Learning Data Sources and Analytics (Doug Armstrong)
Another case study, this time looking at some sources of learning data and some ways it can be effectively analyzed and put to use in a company.
A Perspective from K-12: A View from the U.S. Department of Education on Big Learning Data in K-12
How can education decision makers obtain the increased quality and quantity of evidence needed to fuel innovation and optimize the effectiveness of new technology-based learning resources?
What can be done to ensure that technology-based resources and innovations are up to the task?
How can the learning data these systems collect be used to improve the system’s ability to adapt to different learners as they learn?
How can data better be used to help support the full range of student needs and interests–both inside and outside schools and classrooms–to improve learning outcomes?
How can educators use the system to measure more of what matters in ways that are useful for instruction?
What better support do educators need as they make decisions about which digital learning resources to adopt?
Epilogue: Moving Forward (Bob Baker)
Wraps up the book with another series of questions we should be asking ourselves about big learning data:
What makes big learning data and data analysis important for our organizations?
What does big learning data mean for learning personalization?
How might big learning data play a key role in the life cycle of learning programs?
How does your organization grow and borrow the skills and mindsets to leverage big learning data?
How will the roles of learning leaders need to evolve to foster a big learning data approach in our organizations?
Conclusion: The Future of Big Learning Data Begins Now
We hope that helps you better understand what big learning data is and how it might be helpful.
What about you? Are you currently using big learning data? Do you have plans to do so? It would be great to hear about your experiences, plans, expectations, hopes, and concerns below.
[raw] [/raw] In this article, we'll be talking about workplace fatalities. Workplace fatalities in the construction industry, to be more specific. And to be even…
As the industrial workforce continues to embrace digital transformation to ensure a competitive and resilient future, focusing on enhanced productivity and efficiency to keep operations…
