Artificial intelligence in the workplace: insights into the transformation of customer services

How does the use of artificial intelligence in training affect employee productivity? These and other questions were investigated as part of the long-term research project “ai:conomics” using company data from various large European companies. Initial results suggest that AI can have a positive impact on employee productivity, especially for new employees.

With the release of the free-of-charge version 3.5 of ChatGPT in the Autumn of 2022, the discussion about the opportunities and risks associated with using artificial intelligence (AI) in the workplace has gained momentum. ChatGPT is an example of so-called generative AI, which can generate original texts (such as ChatGPT or Aleph Alpha), images (Dall:E or MyEdit) or videos (Sora) in response to input prompts. Many actors in business and society are enthusiastic about the numerous potential applications, especially because they pledge higher productivity, better quality or greater diversity of products and services. Others, however, fear that such AI applications could (almost) completely take over jobs from well-trained specialists and university graduates. The fact that AI applications rely on large amounts of individual or organisational data, and that the collection and processing of this data is not always transparent, is also regarded critically. These factors can contribute to employees being somewhat sceptical about AI when the question of its use in the workplace arises.

To date, there are few reliable results which document the influence of AI on the world of work and on employees in particular. The main reason for this is the sparse data available, which can also be explained by the fact that investments in technologies have thus far often been analysed in a rather rough manner. In addition, companies that are already using AI, or are currently testing it, have little incentive to report on success or failure and on the concrete consequences of using AI for productivity or employees.

Over the past three years, the Research Centre for Education and the Labour Market (ROA) at the University of Maastricht in the Netherlands has teamed up with the IAB to close this research gap. The two institutes, together with the German innovation agency zukunft zwei GmbH, are conducting the research project “ai:conomics” (www.aiconomics.eu), which is funded by the German Federal Ministry of Labour and Social Affairs (DKI/ AI Observatory). In order to conduct empirical research into the effects of introducing AI into the workplace and to employees, the project team gained access to several European corporations that have tested and introduced particular AI applications.

This article describes the insights gained in relation to the introduction of an AI-supported training programme for the customer-services department of a large financial services provider. First, the quantitative impact of introducing AI-supported training on employee productivity was examined. With the help of qualitative interviews, a picture could also be painted of how employees personally assessed the introduction of AI-supported training.

AI-supported training for the customer services of a financial services provider

The financial services provider employs 152 employees in its customer service department. On average, they answer around 120 calls per month, meaning that the entire department handles around 24,000 customer enquiries monthly. In addition to high service quality, efficient processing of individual calls is particularly important. All employees first complete extensive basic training. Once they begin answering customer enquiries, they receive regular training sessions, roughly every two weeks, lasting between 30 and 60 minutes. The intensity of this training varies with the professional experience of the service employees, with the most experienced employees only receiving occasional training which is specifically tailored to their needs. The training is aimed at improving productivity by shortening the agent’s call-handling time and by improving their communication and problem-solving skills.

Before the AI was introduced, the trainers used information from three randomly selected calls held by the respective employee to develop the training session; this was in order to train the employee individually, based on their personal strengths and weaknesses. However, due to the limited selection of these few calls, trainers and employees found that the strengths and weaknesses of the employees were not adequately tackled in the training sessions.

In order to be able to derive more meaningful profiles, the company introduced an AI in 2023 that transcribes and analyses all customer calls to each service employee and generates a wide variety of performance indicators for each employee. The AI is able to document and evaluate the employees’ conversation topics and communication techniques. For example, the AI can analyse how often the service employees asked clarifying questions, used filler words (such as the word “okay”) or had to interrupt the customers.

Thanks to the AI, the trainers can now access detailed results for each employee and specifically select problematic calls in order to train the service employees individually.

Does AI-assisted training improve customer service productivity?

The financial company did not introduce AI to all service agents simultaneously in 2023. Rather, AI was introduced in a staggered manner to different teams of employees at different times and locations. This staggered introduction makes it possible to accurately analyse the impact of AI on productivity because employees who were trained with the help of AI can be compared with those who were still trained without AI.

The below graph compares the average call duration between two groups of service agents over a period of 9 months (see Fig. 1). While one group was trained with the help of AI from the fifth month onwards, the other group continued to be trained without AI. Before the introduction of AI, there was no statistically significant difference in call duration between the two groups. After the fifth month, when AI was introduced in one of the groups the figure shows a negative difference indicating that the call duration of the agents who received AI became significantly shorter.

In the following figure, the mechanisms of the shortened call duration were examined in more detail (see Fig. 2). The third bar shows that after the introduction of the AI-supported training, customers had to spend an average of 25 seconds less on hold (about 14 percent). The first bar shows that so-called moments of silence were also shortened by 6 seconds (9 percent). The speaking time of the employees was also shortened by 18 seconds. In relation to the average speaking time of 359 seconds before the introduction of the AI, however, this effect is only 6 percent and is statistically indistinguishable from a zero effect.

The results indicate that the service employees handled the calls more efficiently: they needed less time on average to record and process concerns raised by the customers. Furthermore, the time customers spent on hold whilst employees got advice from managers or colleagues to deal with concerns was reduced.

Which employees benefit most from AI-supported training?

In general, both new and more established employees could benefit from AI-supported training. De facto, new employees have little experience and a high potential for error, which is likely to become apparent in the first few calls. However, AI could also improve the training of employees who have been with the company for a long time, as it can identify specific approaches to improving conversation skills more systematically than human trainers could with randomly selected call histories.

The results clearly show that AI-supported training particularly benefits new employees. Their call duration was reduced by an average of 73 seconds as a result of the introduction of AI, while the more experienced employees only reduced their average call duration by around 43 seconds. After the introduction of AI, employees, particularly those with less professional experience, put their customers on hold for shorter periods and had fewer pauses when speaking. However, the results also show that the experienced employees used fewer stop words and their calls became on average more understandable and goal-oriented. However, the average speaking time of the experienced employees was reduced to a lesser extent. The introduction of AI-supported training therefore reduced the frequency of major errors made by inexperienced employees. On the other hand, the training improved the finer conversational skills of the experienced employees.

Personal experiences with AI-supported training

To investigate how service-employees personally assess the introduction of AI as a support for their training, several qualitative interviews were conducted: with four trainers and 14 service employees. The first series of trainer-interviews took place at the time of the introduction of AI in order to explore the expectations and concerns of the trainers and to document their first impressions. In order to compare these expectations with concrete experiences, the trainers were interviewed again a few months after the introduction of AI.

In the first series of interviews, the trainers mainly spoke about initial technical problems with the introduction of AI. In addition, some complained about the increased workload that the introduction of AI initially entailed. One of the trainers commented:

“At the beginning, the AI tool was not 100 percent ready for use right away. It worked poorly and there were either no transcripts or they were too old. I couldn’t coach based on calls from three weeks ago.”

However, in the second series of interviews following the introduction of AI, the trainers were largely positive about the data provided by AI and the way they were presented. The data would allow them to respond more effectively and efficiently to the individual training needs of the service employees. Almost all trainers stated that the AI had significantly improved training sessions. One trainer noted:

“Because I can mark many more calls at the same time with the AI tool […], I can coach in a much more targeted way. I think the quality of my coaching has improved significantly in this respect. And because the AI visually represents what I say, the impact on the customer service employees is greater, which leads to faster changes. If I simply tell them: ‘You use the word ‘okay’ too often’, it can take five weeks for a real change to occur and I would have to repeat it in two or three more coaching sessions. But if I can show it to them visually and write it in the coaching report, you can see a clear improvement just one week later.”

Two of the service employees said they felt that the AI was monitoring them more closely than before. Most of the service employees, however, rated the AI positively. They found the AI-supported feedback during the training sessions to be more precise and constructive. In addition, some service employees stated that they worked in a more motivated way because the AI was able to document and display their progress in conducting calls and dealing with issues. One of the service employees said:

“Ultimately, it’s also great that the AI tool makes it clear to you that the calls are going well. I used a lot of filler words at the beginning, but when you hear later that the AI is noticing that there are fewer of them, you see that the quality improves over time. That’s a big benefit to you personally.”

Overall, it can be concluded that most of the employees interviewed had largely positive experiences with the new AI, despite initial difficulties.

Conclusions

As part of the “ai:conomics” project, it was possible to follow the introduction of an AI-supported training programme for the customer services of a large financial services provider. It was then possible to examine how the changes to training affected the productivity of employees and how employees personally assessed these changes.

The comprehensive recording and analysis of calls by AI enables trainers to identify and address specific strengths and weaknesses of employees, which was previously only possible in a patchy manner based on a few randomly selected calls.

Measured against usual key figures for customer service, it was found that the introduction of AI-supported training improved both the efficiency and quality of calls. New employees in particular benefited greatly from the changed training, as they learned faster and reduced errors thanks to the AI-supported feedback. Experienced employees were also able to optimise their conversation skills more efficiently thanks to the changed training and thus increase the overall quality of service.

Despite initial technical difficulties and concerns about monitoring, most of the employees interviewed rated the AI positively. They particularly appreciated the more precise and constructive feedback which made their motivation and progress visible.

Overall, these results lead to the conclusion that AI-support in the design of individual training content cannot only increase productivity, but also strengthen employees’ satisfaction and confidence in their own abilities.

The case examined here is an example of the introduction and use of AI which is intended to improve the quality of an individualized service. It shows that AI can bring tangible benefits to both the company introducing it and the employees affected. Here, the right implementation in the right area of the company plays an important role. Finding the right criteria for this will be another important field of research in the future.

Reference

Grienberger, Katharina; Matthes, Britta; Paulus, Wiebke (2024): Folgen des technologischen Wandels für den Arbeitsmarkt: Vor allem Hochqualifizierte bekommen die Digitalisierung verstärkt zu spüren. IAB-Kurzbericht No. 5.

Ozgul, Pelin, et al. (2024) : High-skilled human workers in non-routine jobs are susceptible to AI automation but wage benefits differ between occupations.” arXiv preprint arXiv:2404.06472.

picture: Production Perig/stock.adobe.com

DOI: 10.48720/IAB.FOO.20250422.01