Josh Bersin and Arist: The Moment is Now for AI in HR

Transcript:

Ryan Laverty: "...know we've got a few folks still jumping in but we're going to go ahead and get started. Well, Josh first off, just a huge thank you for me and the team, for taking the time. I know that you definitely, you know, have no shortage of things to do especially this time of year. So for folks on the call, you know, we're going to get through a lot of different stuff today. What we're going to start off with. And, you know, we've been working with Josh the Java for a few months now, and they've just been super awesome to work with. What we're going to start off by doing is I'll just have Josh kinda go through an overview of the industry landscape, a lot of different things that we see happening with a I and HR. It seems like this is, you know, such a critical moment for us all. I think us all as leaders are facing so many challenges. So we'll have Josh go through that whole landscape. I'll give an overview of, you know, what we all do here. And we're actually going to give a live demo of a lot of this AI in practice. So much of it is talked nowadays, we'll see how it looks in action. And then we'll have open form and questions. So feel free to, in terms of running the road here, feel free to answer questions in the chat or throw them in the chat here in the Q and a feature, you can chat anything over on the side window and I'll kinda manage those as they come in. So Josh, I will pass the bank over to you. Thanks so much for joining us."

Josh Bersin: "Sure. Well, okay. Let let me spend a few minutes on the background of why AI is important, where it came from and what it means to everybody in HR and L and D… for most of you that are, you know, sort of in the larger companies. And if you're aware of the other things going on in different parts of HR, we're living in an interesting time where the unemployment rate is extremely low… the rate of company transformation is accelerating. 40 percent of CEO'S believe that their company as it exists today will not exist in 10 years, 60, almost 65 percent of them believe they need to spend more time on transformation and less time on execution. And the employees in most companies are burned out. They're quietly quitting, they wanna four day work with week. They want to be more productive, they want to spend time with their families, they wanna raise from inflation. So we have these colliding factors, entering companies where most of the traditional talent models are no longer working. And by traditional, what I mean is in certainly most of my career and most of the companies I've talked to the number one way to grow your company is to hire more people. The more salespeople we have, the more products will sell, the more engineers we have, the more products will make, the more manufacturing people we have, the more, you know, products we can manufacture, et cetera. Well, that's not going to be possible because the fertility rate is dropping all over the world. People are working, you know, trying to find ways, to have more flexible work life. So, from an economic standpoint, every CEO and chro we talk with is trying to figure out how to drag greater levels of productivity, how to reorganize their teams, redesign jobs, create what is often called a skills based organization. Although we call it a dynamic organization. We have a big body of research which is published two weeks ago on the dynamic organization. And AI is a huge part of this. And in some sense, as I've written about before AI has come along to save us in a period of time where we're not gonna have as many people to hire as we have in the past.

So including an HR by the way, now in the middle of all of that sort of business transformation, the HR function has been trying to catch up and what's been going on in HR. We call this systemic HR is a realization that we are not a bunch of administrators doing tasks on behalf of managers. We are professionals. We have careers, we have complex skills and roles. In fact, there are more than 400 job titles in HR. The number went up by 30 percent. In the last five years, there are more than 250. Now there's 250 job titles, 400 skills in HR including L and D and all the things that L and D people do. And so, you know, part of the response to this economic environment is for us to be smarter, more cross trained, work better together as a cohesive whole. And, our big initiative for the coming year is what we call systemic HR, which is helping all of you guys including me stitch together all of these solutions that we build so that they work as an integrator whole on behalf of the company. Now AI comes along. And most of you know this that AI is a 50 to 60 year old technology that's been around for a long time. It started in the mathematics departments of computer science. I was actually a UC berkeley, leave it or not in the 19 seventies in the 19 eighties. I used to meet with the AI guys over there and, they were some of the geekiest people on, in the computer science department because they were working on mathematical representations of data that were very hard to automate because the computers weren't fast enough. Well, of course, last year computers got fast enough. The transformer was announced was introduced and now we have AI everywhere in our lives and, you know, it's not new. It's been around for a long time. AI has been used for credit card fraud detection and other things, you know, noise detection and your headphones and other things. But so now it's entering the world of HR and in particular the generate AI, which is where we can take large amounts of content, typically text but also audio, another can be generated and move and turned into other things, which is what Arist does. And by the way, let me give you a little preview. Ryan, I didn't tell you this. We were on the phone last week, this we Wednesday stay tuned. We're going to be launching our generative AI solution for you for HR people. And it's a very interesting something we've been working on for almost a year. So what this is doing is it's unlocking dozens and dozens of examples of things that you can do that you couldn't do before. And let me just show you maybe, can I show you one or two slides? Is that okay? Let me pull up a couple of things here. And I think the use cases are still being in some sense discovered and developed, by you and corporations and also by vendors. And I would say Arist is very much ahead of the curve. But a lot of the vendors. In the market are to some degree in shock because what happens with AI is it does change the architecture of your software. Typical, you know, typical AI based system is not a transactional system. It's different. It uses different technology and different data. So, let me just show you a picture here here with a couple of things. Okay? So, you know, I've been, I would say, you know, I've become somewhat of an expert on this for two reasons. One is doing a lot of research and we've been developing our own system. If you read through this chart, can you guys see this Ryan did that come through? Okay? We're gonna talk about and in particular, because you're gonna see what Arist does, but there's a massive amount of energy and investment in AI and recruiting to take this complex matching problem between a job or a role or a set of skills that we're looking for. And then a 1,000,000,000 possible job candidates that could apply for that. These are tools like eight fold seek out Beamery fee. And what they do is, they amass a massive amount of data about the workforce, they assess and identify skills, and then they use that data to match it to things that you are looking for in your company.

And, and it's virtually impossible now to run an ad recruiting function. Well, it some form of AI by the way. One of the reasons for that is that on the other side of that dynamic, there are also a I tools for job seekers that will allow a single job seeker to post their resume to 800 job postings at the same time. So, we've got a war for recruiting on the candidate side against a war for selection on the employment side, lots and lots of things to do there. And career mobility, all the opportunities you have in L and D to find a developmental opportunity for someone to move them to a new role to give them a better career, to show them the adjacent role to teach them things that you think they need to know, but they don't know they need to know. I has been very successful in talent matching and talent marketplaces, in creating element plans. We actually have a career navigator for HR that we're going to launch next year, that uses AI that if you're a member of our academy will show you based on the job, you're in the skills you could learn to get a different or more advanced job in HR, just in the domain we're in that gets into the issue of job design. A lot of companies are flattening their organizations as just with a bunch of companies in New York last week, coke cola, a walmart, many others have flattened their job architectures. So they don't have so many levels which gets in the way of people moving around and they use AI to look at comparable jobs in different parts of the companies, so that they can collapse them together and simplify the job architecture, reduce the amount of layers, make the team more efficient in the area of self service. You know, AI is very powerful for chat box answering questions. In fact, this is what you're gonna see some stuff from is that does this? I'll let you, I'll let Ryan talk to you more about ring and development and rewards.

You can use AI to assess pay inequities, you know, pay equity is a big topic in the comp side of HR. And what happens there is you do statistical analysis on pay variations and you look at pay variance versus other factors. And if you find a strong correlation between variation and pay and race, you're gonna get sued. If you find a variation in pay between pay and age, you're going to get sued. And so companies are running around trying to figure out how to analyze all this data. I can do that very effectively in the area of employee experience, understanding data that comes in for employees, analyzing textual feedback, analyzing conversations to see what the problems are people are having, analyzing turnover, looking at job fit in leadership. We now have, I now know several vendors that are using AI to look at a large corpus of data about your employees, assess their skills using AI, which it does in a certain interesting way and then look at the leadership capabilities in the top employees, say, director and above, and compare them to each other and compare them to other companies that's a really interesting fascinating opportunity that's going to be opening up. And then in performance management, which I just published a pod cast on this yesterday, we basically have to assess people based on lots and lots of data. What were the job projects they did? What was the feedback? They got? What was the poor performance they had in different roles based on business metrics. We're going to be able to use AI for that. So, there's millions of they're literally millions of applications. Now, one thing I would say about it is, you know, there are tools like Arist which are revolutionary in the use of AI for developing content and serving the needs of employees.

They can create in a sense automatic teaching assistance by the way the AI that we're launching in is going to be essentially an information assistant for everything in HR, and you'll be amazed at what it can do and what you will be able to do is put information into a, into an AI engine like Arist, to develop either a course or a chat bot or some product that people can use. However, what that means is you need to know where that content came from. So, I just got off the phone with a whole bunch of chro, a little bit earlier today, data management, data quality, data author authorization. If I stick my compensation guidelines into an AI chat, but, and I produce it and turn it into either a course or an interactive experience for employees who's allowed to see it. Is the data current is the data accurate? Who authored it? How do we audit it? There's a lot of data management things that you're gonna want to think about as an enterprise on how to do this. But that all said, I would say the biggest obstacle most companies have to be honest is not knowing what it is. I just got off the phone with these, you know, a lot of senior executives in HR and they're intimidated. They don't understand it. They don't they're a little bit of afraid of it. People that don't understand it, tend to push back on it. They're worried about it eliminating their jobs. I will say on the topic of jobs, this is like every other automation tool that I've ever seen in my career. Some jobs will go away. I mean, if you're an administrator and you're taking the phone, answering questions on how somebody can fill out their benefits forms. You know, I don't think that job is going to be needed, but that person can now work as a more strategic customer service agent or an adviser for you. And L and D. If you really get off general rating videos and audios by hand and it starts to become automated. Your job is probably going to be more fun and more interesting and more creative than it was before. Because now you have this massively interesting tool set to make content even more compelling and, the sort of the drudgery of the initial stages of the content development process can be automated for you in advance? I remember, you know, I'm a little older than a lot of you. I remember when the first IBM PCENTE landed in the office at IBM, I distinctly remember the exact location I was when we opened the box and we turned it on and we looked at it and we were all staring at each other. Like, what are we gonna do with this? And that was a period of time when people were using word processors. I mean, manual machines, not software. They were typing things by hand. We didn't have an it's any spreadsheets we were using calculators, and look at where we are today. The unemployment rate is three point eight percent. It's the lowest it's been in 55 years. So, a lot of jobs get created around these tools, you know, forcing all of us to learn how to take advantage of them and use them and decide what we're gonna do next. Just one more point in the area of learning and development. In a sense, the problem that we're in an L and D is we're really a combination of a performance consulting organization and a content organization.

We get a request for a course or a program or a new like one one of the things by the way that I'm sure Ryan will talk to you about is courses about AI. We built a course with Arist, about a I that we've been giving away for free, which we encourage you guys to take. And I'll let Ryan show you how to get to it. You, you should be spending as much time as you can on the consulting process of land and on the measurement and return on investment and the continuous improvement of your content. The actual process of building the content as interesting as that seems, using dream work, river or whatever tools people use these days. If that gets automated, more power to you, think about some of the new problems you can go after the faster content programs. The faster you can get things to market, the better you can do of analyzing the skills and development and the results of the programs you're developing and the opportunity you'll have to be more creative and use new tools to come up with new experiences for learners. So… I'm not afraid of this technology at all. I think it's intimidating if you haven't been paying attention to it. But what you're gonna find a year from now, this is going to be fairly common place. You're gonna take it for granted and you're just going to be using it and that will be further and further advancements in all the use cases. Okay, Ryan, how's that I'll stop there for now?"

Ryan Laverty: "I think, that's a great overview, Josh. And one thing, one thing I'll just highlight and then ask you a question before I jump in is, you know, I think there's this whole theme of, hey, AI was gonna take our jobs. AI is going to change everything and I think those are two really different things, right? It's not gonna take all of our jobs like you mentioned with the PC, but, it will change a lot. And so, in a moment, I'll kinda give an overview of Arist and some of those capabilities first. Just a question for you. You know, if I'm an, and leader and HR leader, how do you think my role personally is going to shift because of, you know, things in an AI specifically?"

Josh Bersin: "Well, I think if you're an L and D leader and you're managing a lot of resources in L, and you've got to take a really serious look at where you're spending your money on what. And it is probably we have by the way, we just launched a course in the academy called the learning and development super class and it's a really extensive course on how an L and D leader would think about their organization to structure their land organization to take advantage of changes in technology and tools. So, I, so I think as an L and D leader, it's a good time to think about how can you reorganize and re engineer the team? So the team can take advantage of these tools and get more work done and reorganize what people's jobs are if you're an L and professional, and you're a content developer, a teacher, a trainer, these are incredible tools. You can use a tool like the one that we have or the one that you have, and you can load content into it and you can provide content to your, you know, employees at a rate of speed that you never could before. And that means you can develop more content. You can iterate faster. You can be more creative and I think some, to some degree it's getting to know products like yours. I mean, Ryan, when I first of all, I thought, I always thought Arist was great just because of the mobile experience that you guys create. But when I saw your generator, your AI generator, I thought my God, you know, why don't, we use this all the time. So we're going to be doing a lot more with Arist in our academy because it saves us a lot of time to generate content and we can generate content in a format that is, very compelling and very easy to consume in the flow of work. Doing that by hand is a massive amount of work that most companies couldn't do anyway. So, so I see this nothing as learning and performance improvement for all of you guys. So."

Ryan Laverty: "And I appreciate, you know that, Josh. I think I'll share my, you know, screen in a second show something. I think one thing that really was an interesting conversation we had too was when we were just when we were talking about this for you all. And even at the JP, you face a lot of the same challenges with some of these learning leaders. It was like, hey, we've got this stuff. It's gonna come out, you know, next week or right now we have to invest a bunch of resources to go build all these things then to go market it to all these people to go take those things then to go figure out how to did, right? And so, that whole supply chain of L and DI think is just getting."

Josh Bersin: "We live, we have lived the pain of all of you guys."

Ryan Laverty: "So you can empathize with the folks on this call more than anyone, I think. So. I'm just gonna share my screen. I'm gonna give a, you know, for folks who've seen, this is going to be, you know, the world's fastest overview. I see a few questions popping in. Just throw those, in the Q and a function as we go in. And then I'm just gonna give a quick crash course here and we'll get to questions afterwards. And so."

Josh Bersin: "Basically for those."

Ryan Laverty: "Who don't know, you know, what we do, Josh is kinda hinted at it. What Arist really does is we bring learning where people spend all of their time and so that's learning nudges, and communications really just directly in the apps that we use every day. And so we work with, you know, about 10 percent of the fortune 500. And what this actually, you know, comes down to is I'll show you what this looks like in a sec. But what we find is that folks spend about, a, you know, once every month or two months ago in the LMS or XP, they spend about point two percent of their time in one of those systems. But more than 80 percent is actually closer to 90 percent of all a synchrony time at work. We're not on Zoom is spent in something like a messaging up a slack on Microsoft teams, SMS, or emails and so to create outcomes and a lot of things create powerful learning outcomes. But where we're going to see a 10 1,520 X difference is really just by meeting people where they are. A, one of the first articles we actually read years ago, Josh, when starting the company was an HR article and you had, a quote in it where you said, you know, the average, when you terms of learning the flow of work, you said the average knowledge worker spends and dedicate about five minutes per day but no more to any type of formal learning, right? And usually that's a one time instance that's kind of been averaged out over weeks or months. And so that was a lot of, that core premise."

Josh Bersin: "I think it's even worse today."

Ryan Laverty: "I mean, now we're always."

Josh Bersin: "In the flow of work, every minute of every day."

Ryan Laverty: "Even you're right? Even on whenever you meeting is a Zoom meeting, you're always in the flow of work, right? So even."

Josh Bersin: "Meeting you're looking at your phone…"

Ryan Laverty: "Exactly. And so I think, you know, for us again that creates the size make shift of what we actually do is we're bringing learning right to where people spend all their time. And so, you know, this is an instance of Microsoft teams. I'll get this like I would a Microsoft teams for those who are more familiar with the phone version. This is where, you know, and this was actually I'll show side kick in a minute. This is created by side kick AI. And so we can upload a, you know, a white paper from JP Morgan Chase on kubernetes, it'll break this down into some of this content. But in terms of the model itself that we're referring to when we talk about Arist, I'm getting a few minutes per content per day either as a instant or spaced out over, you know, a few days or a week or two. But it's only taking me five minutes per day. And there's no new app downloads. There's no clicks to get to any access. There's no log in and there's no change management for employees and for learners because they're just using a technology that they've been using for, you know, years or even decades for some folks. And so, I think what this really comes into for us is there's the really these three big components that I'm gonna show you a sec, how AI changes a lot of this. What we're really doing when we talk about, that supply chain of land is, you know, let's say a new product gets launched or something new happens. You as an L and organization have to be able to move as fast or faster than the business can or we're going to get left behind, right? And so what that looks like for us is, you know, your new product gets launched, goal setting season comes out for managers. You can instantly use AI to create courses. You can push those into systems people use all the time, I, into messaging apps or let people, you know, pull things, scan a QR code or you can allow them to, you know, or even text a code to a number which I'll again show you with Josh 's course at a moment. And then finally where this is really important is because that course is spaced out over time. All the data we collect is spaced out over time. We're collecting about 100 times as much data on any individual learner as any other system. And the reason is because folks are interacting every single day. Something Josh had asked me before is how many courses do the average learner take and people get spam, by messages. And the conversation we had was, well, the average person might watch an hour long video once every few weeks. The average person answers about three to 60 Microsoft teams messages if that's your system at work every single day, right? And so to spam folks, we'd really have to overwhelm the change is really there and for them to interact with the system like that every single day that change management is already embedded in the organization. And so that size make shift is that this can all be done and pushed out in minutes, not in months. And it's far more effective. Really what we come into now is really just the idea of where I can help Josh to your comment earlier. I threw this in there. No, it won't take our jobs, right? I think that if we think about those three components of creating that, delivering it and measuring it, if I'm a learning leader, you know, first, I'll let a, I build the first draft. And so I'll upload a swarm file, a video PDF file. It'll convert it into this type of format that fits in that messaging app. And again, we'll show you in a moment what this looks like. But I'll prompt the system. It's fully closed loop. It won't learn any. It won't you know, use the company information to teach the larger model, but it will, you know, become pretty smart with just contextualizing the info you've gotten and the type of info that comes back second. Once I've got all that done, I can say, okay, now, I'm gonna put timely learning on autopilot. And so I've created my course really quickly. I'm gonna now embed this with systems like success factors or workday or plug into something like a teams are slack. But I don't again, I don't need to integrate it if I not required and I can actually make it. So I say, hey, are, you know, sales in our sales division, these 300 folks just got promoted to a new sales manager. I'm gonna automatically prompt a course to be, you know, sent to them on this new product that we pushed out or on the, you know, fundamentals of sales management. And really what we're doing is putting timely relevant learning on autopilot. And so the role of the, you know, land person changes from creating content and marketing content all day which is about 80 percent of and professionals job between those two things into actually just rapidly, you know, getting adoption, spending a lot more time on analyzing data and on strategy and needs analysis than on just creating and on measuring. And the third piece after we've created things rapidly and push those out to folks is that with all that data, I can predict a lot of challenges that haven't happened yet. And so again, we've got about 100 times more data on individual as another system might. And so we can actually go look at and analyze responses. Again. If I've got a sales team management team and I can say, hey, we've got, you know, 6,000 folks in North America. They just went through, a quarterly review process. They just went through setting goals for the year and we had them define how they felt about their careers. We actually think we're going to have a turnover problem in the southwest because of this information, right? And so, I think we're moving to a place where L, and leaders, HR leaders can be much better partners to the business because you can actually be predictive and strategic with all this type of information, to, and the last thing here before I'll you know, jump into side kick is really just this paradime shift that, you know, what your CEO can expect from learning are things like, hey here's, how we're seeing our customer reps, you know, respond to different rates of change here's. How we're seeing a huge turnover risk coming up based on four or manager is we think that this product launch is going to go well or not go well because sales reps in North America are still struggling with these competencies, right? And so, regardless of what the specific and HR function might own that size, make shift again is really just being able to, instantly, you know, learn of skill and understand all of these things and actually be predictive partners to the business because we can do all of this so much faster. And so, again, there's a lot. I know that was a super quick crash course. There's a lot of talk in the space I pull this up. And, you know, Josh, we can chat a bit about for you all what this was like. What I'm gonna quickly do is just show you all what side kick is like. And so what I'd love to do, I'll give an example here. If folks want to throw into the chat something that you are, you know, trying to teach right now, maybe a course that you're actively building, let me know and I'll prompt it live just to show you that there's no special Andy work going on behind the scenes. I'm gonna show you how I prompted this course. I'll show you in a minute. And again. While I do this, feel free to throw any topics and chat and I'll try one or two of those. But here we're going to use paxlovid as an example. So, paxlovid is a certain type of drug the visors administering to COVID patients. Let's say, you know, administering paxlovid to patients is what we want to learn. And then we're gonna say, you know, the audience is patients who have COVID or long COVID. And then the purpose might be something like, you know, understanding what paxlovid is and how to recommend it to, you know, to patients. So actually the audience here would be patients, it would be caregivers administering… a loved to patients who have COVID and I see a few things coming in the chat. I'll use those in a sec. And so what the system has done is it's grabbed all the info from the PDF here. And now we're going to generate this outline. So again, this will take about, you know, 10, 15 seconds. And so first, it's getting into introduction. It's going through the key limitations of this. Again, it's not pulling this from the internet. It can pull info from the internet but it's just going off of this PDF. We've given it. Second, it's understanding drug implications. Again while this is generating, this is pulling both from, a, things that are, you know, the system uses like an open system, but then it's also, it's gonna close loop where it's learning a lot of this company information. And so we'll actually get to the place where you can train an internal system on, you know, a lot of company information, use that as a learning partner. But then you're also going to get info from the outside, but it's secure because you're not actually, you know, training info from the outside. And so you see, we can see here that it talks through addressing special cases, handling it versus reactions that's a lot of what this is really important. If you go through the PDF, navigating dosing and prescribing. And so it looks like it's done a pretty good job. We're going to go ahead and generate course. And again, this will take about three or four minutes. I prompted this right before we jumped on. Just so you wouldn't have to wait. And again, all we have to do is come in and add images, but we see this outline is pretty similar because again I prompted it the same thing. It's you know, it's talking using a bit of different language but covering the same things. And then if we come into our lessons here again, what we see is it's not just giving me, you know, some sort of blanket information. It's gonna talk about this. And then it's going to prompt me with different scenarios. So I might say, hey, when treatment is done, when is the best time to do this? This is checking my understanding. It also because we've trained this on over a 1,000,000 different courses and on, a positive learning outcomes. It's also gonna prompt me with a scenario. Okay. We've taught you this knowledge. Now, we're going to prompt you with scenarios to test the application of this knowledge. So the system is pretty smart and doing a lot of that. But I think to josh's point earlier where this really goes as we can say, okay, I, as a learning leader can actually keep up with the business as soon as I know that there's going to be a training need. I can get something out there right away. And so I see a few things in chat. I'm gonna go ahead and prompt this in here. And I also see a hand raiser two for folks just because of the number of people on the call. Try to use, the question and answer feature. And so let's go in here and then prompt one more. So let's see here… how narrow science can be better manager measuring productivity, customers don't use our products. There's a bunch of great ones in here. Let me see. And again, we can prompt it with something really simple like change management. We can prompt it with something really specific like understanding leadership principals for employees at Amazon who are mid level manager, right? So let's see here change leadership skills, employees need to support a company transformation. We'll grab that one just because of how it's worded there. So again, I'm gonna say leaders and we can be competitive here. It's pretty smart leaders supporting a company transformation. And maybe I'm gonna say the purpose is, you know, understanding the essential components of change management as my organization undergoes a massive transformation. And how I should, you know, manage that change by company level. Right? Again, it's just kind of out of the box but if we did, if we did want to paste the link in here, we could also paste a web page and pull that in, a, again, I see a bunch of questions about the tool itself. We don't have time to answer all those today, but we'll include kind of an e-mail after that, you know, you can reach out to and we'll get all your questions answered today. And so basically, it looks like we're going through the basics of the team process, knowing your team keys to successful transformation, communicating change that's huge, right? So how do I engage stakeholders effectively? I'll let, this is going even faster than I can then I can describe it. I'll scroll up in a moment here recognizing change, celebrating success. So we see communicating change, engaging stakeholders that's huge navigating resistance and then just leading by example. And so, you know, again feel free to, you know, throw in the chat, how you think this? Did I'm going to include a link for everyone afterwards? We'll e-mail everyone with a."

Josh Bersin: "Ability."

Ryan Laverty: "For you to try this on a web page yourself? And so for folks who I didn't I'd love to, you know, prompt all 1,520 these I see. For folks who didn't get to try it, will include a link as well. But Josh, I'd love to, you know, I'd love to kinda pass it back over to you. So, you know, I know you mentioned that, we saw this happen and we see this go through. I guess first, how do you think it did on the change management course?"

Josh Bersin: "I'm just blown away every time I see this Ryan… I mean, this is an example of a highly optimized tuned LM based system designed for this form of learning. That you can see in it exemplifies what happens when you apply a carefully to a very specific problem. And, I totally agree with everything you said. All of these programs have to be developed quickly because there's a time urgency, to the problem. This is a fantastic solution. By the way. Let me go back to something. You skipped over a little bit though there's a concept. I don't know if a lot of you know this concept of space learning, but there's been science for years that says that. And this was actually originally prod university did this and they looked at the teaching of anatomy where you have to learn about every part of your body. And what they found was that more time in the course in the class, getting into more detail. There was a complete tail off in learning. There's only so much people remember. There's a very quick forgetting curve. And one of the ways to prevent the forgetting curve is to force people to use the content and ask them to use it in some sort of a problem. But an even better way is to space the learning out. So they learn something then they don't talk about it. Your brain sort of absorbs it but you sub consciously are not aware of it and then bring it back a day later or two days later and reinforce and remind yourself of that topic that's called space learning. And there's a couple of really interesting studies which I can point out to you guys if you want to look at them that show you the retention and the longevity of your learning from space learning. This is what do a lingo tries to do and other tools is bringing you back to the content to remind you of it. And then your neural pathways get, you know, better set, to remember the concept. So not only is this an amazing content generation tool, but the way it delivers content is also really to me revolutionary. So there's all sorts of great things here."

Ryan Laverty: "Absolutely. And Josh, one thing I'll just I'll add some context there. So, we have done a comprehensive study with the university of Washington and they took a bunch of OB gyn who are going to go through their annual pro exam and they took a huge group of folks who went through traditional learning methods. And then they took a huge group of folks. The only difference was they said we're gonna use errors. We're going to do just space learning. We're gonna do reinforcement. There was a 12 percent bump in the complex medical exam scores. This is again, this is not just a simple thing to learn, right? A 12 percent bump. And so for us, what that means is people are getting better care. They're understanding this information better, right? And again, most courses are spaced out over one to two weeks. And I think for us why that's so powerful is, you know, even in the workplace, if you're a manager, and it's something like giving and receiving effective feedback that's not a one time course. I'm coming back and I'm saying, hey, how did that go? Did you try this method? Let's try this again. Let's go back to your action plan that we created together. Let's try the scenario, right? And so it's a lot more in depth because that's how learning things in the workplace is it's not just receiving a bunch of info. All at once. It's a few things really well practiced to get to mastery. And that's really what that whole model is focused on. Josh. I'd love to, you know, shift gears a little bit. And actually, while I jump into some questions here and folks feel free to throw a Q and a and chat. I am gonna pull this up as well. And so, Josh mentioned the course that we created together on locking the power of AI and HR. Again, this was created by side kick and we added images and some fun emojis and things in there as well. And so if you text AI now to two zero, two zero two or you scan the QR code, you can jump in and get started. So I'll leave this up for a moment or two and we'll send this to folks after Josh, you know, one, one big question that I have for you that we received from a lot of leaders this year especially is right now they're hearing kind of two different perspectives all at once. They're hearing, hey, I was going to change everything, we need to be ready for change. We need to change really fast or we're going to get left behind and they're hearing especially in HR and L and D. Don't touch anything. It's a down budget. There's no discretionary budget like just don't enact any change to bringing anything new. If I'm an land or HR leader, how do I, how do I close that gap? How do I navigate those two different perspective?"

Josh Bersin: "Yeah. And, and I think this is the sales challenge essentially for you guys. But I've seen this for years when there's a pair time change. This is a paradime change. So you may companies may have to spend some money to invest in this new technology, to see rapid and accelerating changes and reductions in spending later. So, you know, if you talk to somebody who's budget is full and they say, well, we don't have time to invest in a call us in a year. Well, a year from now, everybody else has gone up this learning curve and you're still here. So I'm not saying you have to, you know, redesign your whole HR, an L and D budget today, but a tool like Arist which is not a super expensive system could teach you what's possible and give you the opportunity to do some new things and reinvent your business processes in a way that I would imagine in a few months, you're going to save money just in terms of time to market and content development costs, or things that you might be outsourcing, to a vendor. So, I just think you have to invest in this. I mean, I spent so, you know, we're not a super big company. We're we're about 40 employees. We started working with you guys about a year ago. That was transformational for us. We started building our own AI about nine months ago and it was not a huge project but it taught us a massive amount of what we needed to know. So we could build something that would scale. And this is a learning curve that you're going to go down at some point and I think the ROI is going to be massive. So, I think for the companies that say, no, we can't afford it. They need to take a step back and, you know, kind of reset their priorities."

Ryan Laverty: "Absolutely. And, I think one of the other big questions when it comes into priorities is just, you know, what we hear from a lot of folks is, hey, I think this is just going to be kind of a like a big change management shift, right? It's just a, you know, something you mentioned to me on a past conversation we had was, it's it involves changing the way of thinking on a few different fronts, right? On content generation, on delivery and learning on content being spaced out. How do you think, you know either Arist or just things like Arist are going to end up working with integrating kind of the end."

Josh Bersin: "I would say of all of the domains of HR that I've had a chance to work with, I think land is the most able to adapt to new technology. Go try to do this in recruiting, go try to do this in performance management. You're going to have a two year project?"

Ryan Laverty: "Of change management. It's way harder."

Josh Bersin: "And I think you guys are used to playing around with technologies, learning about new media, exploring new technologies, attempting to test new tools, and you can try it. You can leave all your regular stuff fine and you can use this on a new program or I can use this on an experiment, use it on the AI program. You'd like what we did, you know, look at the course we built on a, I take, give it to your employees and then add to that and build your own courses on, you know, what you think people need to know by the way, I've had a lot of companies come to me in the last just few weeks and say we've been tasked on building an AI awareness course for our whole organization. Where do we start? Well? I mean, I think, is a great place to start."

Ryan Laverty: "And we'll give you an outline for free too that's the, and, I think that's a really good point because so, another question I've got for you joss. I just, I think there's this other debate where a lot of what this brings to learning organizations. And just the these types of technologies bring is speed and agility. And then that's something we've seen just be really, kind of undervalued, in the L and D space, broadly curious for your perspective on what role you think speed and agility plays in the L and D and HR space in general. And how do they balance being fast and agile with being, you know, thorough and effective…"

Josh Bersin: "You know, we did a, about three years ago. I think it was right before the pandemic, we did a big study with it on the, what we eventually call it was the adaptive learning organization. We, all of the earlier research we had done was called the high impact learning organization. And it was about structure and roles and content strategies and infrastructure and stuff. And when we relooked at a lot of that data, on lots and lots of companies, what we found was that the companies that were the L and D functions or departments that were adding the most value were actually the ones that were the most responsive. They were able to build things quickly. They were able to iterate, they were getting closer and closer to their customers. So they knew what the problems and the work. You know, if you don't know the work environment of your employees, you can build a course that they don't ever have time to take. They don't wanna take. And that agility, what we ended up calling the adaptive learning organization became one of the biggest drivers that was different than it was 15 years ago. When I looked at this before, it was sort of organization and structure and governance, things like that. Those things are still there. But, you know, time to market and quality of experience are kind of everything now and instructional rigor comes, you know, sort of a little bit lower. Yes, you need to get people to learn the right things to do their jobs correctly. But, but if you can attract them to the experience, then you can't get them to that point."

Ryan Laverty: "Yeah, absolutely. And I'm gonna for folks in chat, I'll jump to a few of these, you know, these questions here. Anything that's Arist related? I'll have, you know, folks from Arist message in the chat just while we've got, you know, what? We've got others on the while. We've got Josh on the line. Josh. One question here that I thought was really interesting, you know, is where do you see the battle lines for ownership being drawn between clients entering often times priotary data and, you know, expand outcomes versus provider org who kind of want to encroach on the ownership? Any, any thoughts there?"

Josh Bersin: "Well, you know, from what I know about the market, roughly 70 percent of the content you deliver is internally developed. There are standard courses on, you know, how to do software engineering and how to do it security and leadership development, and things, you know, that you can buy off the shelf. But the most valuable content that most companies have is the stuff they build themselves because it's very domain and industry and company specific, it's cultural, it's aligned to real initiatives inside the organization. And whether you use AI or not, you own that. If you work with a vendor that is appropriating or reusing your content without your permission, I wouldn't work with them, right? I don't think you're gonna, I don't think that's where this world is going. Listen, the idea of an open chat, CPT is great for the general population. But inside of a company, most of these systems are exclusive to you. So, you know, I know the B to C market is very attractive to software engineer, software companies. They wanna do B to C because they think it's gonna get really big, but it's very different from B to B. B to B to B systems have to be data protected and secured. So, if I understand the question, I think you're gonna own your content and I think you don't want to let other companies use it. Some of it's proprietary. Anyway, in terms of, you know, HR, outsourcing firms that build content for you. They usually are very explicit about whether they're gonna share the content. And there's legal language in the contracts that prohibits them or enables them depending on how you want to do it. I just wouldn't stick your training content into chat CPT. If you think it's something that's exclusive and proprietary. I don't know where it goes to be honest."

Ryan Laverty: "And that's the thing is, it does usually go train that larger system, right? I think for us when we, so you talked before about an adaptive learning organization and we've been, you know, big on this idea for a while as well. But for a while we were, you know, telling folks hey we were using the GP T, three plugins, these ais and folks were just kind of really unsure, right? We had to wait for a lot of technology to catch up and folks kinda came back and said, you know, we had some organizations saying let's go. Let's jump in, with both feet in the deep end. We had some organizations we work with like large governments, fortune 500 organizations that are saying, look, we can't touch this at all. And I think what you know, the balance is you can have both, you can have a closed loop system that investor information understands it and we'll split it back in your instance, but it can learn from the outside and take things from the outside without pushing your data back to the outside. And I think that that's where that paradime shift becomes really important."

Josh Bersin: "Here's the link to the adaptive learning organization, research by the way, if you guys want to read it. Yeah. And, and, you know, I think the other thing about using the open sort of LMS, I just learned something interesting last night… they can't purge the data if you go to, if you inadvertently give your content to an open LM and later sue them and tell them you want it back. I'm not sure they can delete it. Yeah, because the system, they have to retrain the whole system from scratch. So this came up, actually, the sec is start or, the FTC is starting to look at data expiration rules. There's data expiration rules in Europe. And once something's trained in the LM, you can't expire the data. So, you know, I think your it departments and your security people are going to be more and more suspect of you guys using these open internet based, you know, general AI systems that's certainly where I see it going."

Ryan Laverty: "Yeah, absolutely. I know some of the folks we work with are in defense, financial services. They can't they're like, there's no way I'm going to get fired if I use an open system, right? And it's just, it's completely unimagineable Josh. I know you know, a few other questions coming in here. So, I think that again, for the folks asking air specific questions, I'll kinda take those after one other thing I wanted to ask you about really just comes down to kind of the role of change, managing AI within the organization, right? Do you think that there's opportunities for L and D and HR to kind of lead that? Are there other ways?"

Josh Bersin: "You, every single person on this call is going to build some course about some."

Ryan Laverty: "I thing and…"

Josh Bersin: "How people should use it. You're going to be teaching people how to use AI in the Salesforce, how to use AI or, you know, it won't be called AI. It'll have some name. And I would say the biggest learning experience I've had as a user and developer of an AI system degenerative system is the answers are not always predictable. You'll get different answers at different points in time. And so you'll be constantly, you know, kind of revisiting, the system, and learning how to it. And the second is it's an iterative development process because the technology is changing so fast. I'm sure you see this at Arist, an open a, I just unleash last Friday Monday, what we say it was a huge varage of new features. Yeah, I'm sure you guys are, you know, our platform providers doing it too. So, so you guys are all of a sudden saying we've got 100 new things to evaluate here. So these are not, you know, you buy it, you roll it out and then you don't touch it for two years. It's not like that."

Ryan Laverty: "100 percent. And I'll you know, I'll give you an example and so, you know, what the folks that released last week was around training a specific instance. And we just talked with my co founder and chief product officer and she said, yeah, the, you know, the team is already on it and it's you know, we can plug it into these systems and everything. But again, I think, where we're going is all these startups that were building themselves is where the AI for this specific thing just got disrupted. And what we can do with learning expertise is to say, hey, look, we can actually just embed that capability in a closed loop system within your instance. And so a feature where you go in that classroom. And instead of just building a course, you can, you know, hey, I've uploaded, you know, 20 PDF into this closed loop system. I can now ask you questions and it's gonna come back with things about my Salesforce, right? And I think that that's where this stuff gets really interesting because you actually, you don't just have something where you've got to prompt it all the time. You have kind of like a knowledge expert on your team, which is what you were talking about before with, you know, with what you all are doing with all HR is just having someone who has this, having a system that has this."

Josh Bersin: "In our system, you know, which is interesting. We've learned so much about it because I've gone through the, what I call them as the prompt chains, the chains of prompts that people tend to path through. So if you're working on employee experience or training, you're going to be able to see what is the pattern of interactivity from the average user, and then predict those sort of past as opposed to in the traditional development process. You've got to build a bunch of chapters and just hope people go through."

Ryan Laverty: "Them. Yeah, right."

Josh Bersin: "Yeah. So you have much more flexibility now?"

Ryan Laverty: "100 percent. I think one, one other thing to Josh is really just around, you know, what do you think is kinda next? I know when we talked, you showed kind of this slide of here's this evolution, of what learning has gone through. But if you had to kinda put it in simple terms, you know, for an HR leader one, what do you think, is that big next step that's gonna come first? But then two, how long do you think a lot of this transformation is actually gonna take for some of these organizations?"

Josh Bersin: "Well, you know, it's funny. I sometimes I'm ahead of my time. Sometimes I'm behind many years ago, at least five years ago, I started to give speeches on learning and the flow of work. And at that point in time, there was none of this new technology. And what I was trying to explain was that we have to find a way to make the content experience relevant to this individual person's. Location and point in time in their understanding. If this person is a neofite, don't, give them an advanced course and make them go through chapter one and sit there scratching their heads about what this is, if this person is an expert and they want examples and case studies and exercises, don't make them go through the first 15 chapters to get."

Ryan Laverty: "And…"

Josh Bersin: "At the time, it was impossible. It was very hard to do that. We didn't really have any flexible tools for doing it. I never even dreamed of the day that the course would be delivered on my phone in a message. And eventually, you know, based on whether I responded to the message or not, it would give me another message that was relevant to that. But we're basically at that point. So I think where we're reaching is more and more in the business community, we're going to be able to put together programs that intelligently learn what this person is trying to gain out of this course. Some people want the basics. Some people want the advanced stuff. Some people just wanna look up one topic and then be done with it. And I think the content systems are gonna be smart enough to figure that out. It's it's to some degree a little bit analogous to where the advertising market goes, where we get very pinpointed advertisements. And I do think our phones listen to our, sometimes our conversations and somehow get smarter. You know, if I can fast for two or three years, I think that this is where it's gonna go. This is, this is where you guys are going. This is where Microsoft's trying to go with their all of their viva' stuff. And I think that, the training systems that are using the old paradigm are gonna become not irrelevant but just lower and lower percentage of the content that we deliver to people as these, you know, more dynamic systems are more widely used. Yeah, absolutely."

Ryan Laverty: "I think that, you know, one of the things that when we talk about, you know, you mentioned with some trends kind of is how trends are right? You hit them some, they come back. I think the thing we always look forward to, is it adding real value to the business, right? We're always working with these leaders who are trying to figure out depending on the culture of the organization. How do I make a business case for things like this, to my leaders? And we'll?"

Josh Bersin: "The business case to me is, I mean, I, I've talked to so many heads of L and D over the years. I don't care how great your course is if only 40 percent of the employees even take it and only a third of them finish it. You've had a limited impact, unless you were sure that those were only the people who needed it because, you know, maybe it depends on what the course was designed to do. So, if we reach more people in a more relevant form, the ROI is gonna go up and they're gonna say they're gonna tell, you know, ROI analysis isn't as sophisticated as some people think it is. They're going to tell you this was a great course. I learned a lot. Please give me more or you're not gonna hear anything at all if you don't hear anything at all, you kinda need to worry."

Ryan Laverty: "And I think there's also a case two we've seen is just being able to go back to leaders and even just say even within the L and D organization, like a lot of we've talked to some CEO'S about their land organizations and they've said, hey, I, you know, I don't think they respond to needs fast enough and I consider how competent they are to be. How fast can we, you know, just meet some of these coming needs for the business? Josh. We've got two minutes left and, you know, thank you so much again for your time. Last question for you is just, we've talked about a lot of different stuff around AI and HR and land, any final kind of parting words of advice to land leaders who are going, into the thick of it with, you know, changing dynamic organizations, brand new tools, and all this other stuff going on."

Josh Bersin: "I think this is, I think you guys, I think you guys and, I consider myself to be part of this. I think we have entered one of the most exciting periods of time and a long time, I really felt like the L and D market was very stagnant. We had the birth of the L XP five six seven, eight years ago and then very little VR was great but it's a fairly niche technology and there wasn't much else happening. Now, you can completely reinvent what you're doing. You can develop content faster in new forms. You should jump into this, get the, get a little bit of investment from your organization to go out, use a tool like Arist, get your hands dirty with it. And I think it's going to revolutionize the space. And I think training and development is going to be one of the biggest applications of AI and other forms of development like a chat, but that supports people's. Learning are just going to be massive. And like I said, I think we had a lull in this industry where there wasn't as much innovation for a while and I think that's come back. So, I think it's a, very exciting time."

Ryan Laverty: "Couldn't agree more and couldn't say it better myself. Well, Josh, thank you so much again and we'll see you all soon."

Josh Bersin: "Thank you Ryan."