Implementing LLMs in our internal mobile apps: Design, Demos, and Deployment

This is a summary/blog of my presentation which I recently gave internally at SAP during the One SAP Mobile Summit. The event brings some of the best and likeminded SAP mobility related groups and teams together during a week long, in-person, and virtual summit.

This is another blog post of a series around the enterprise mobility IT team at SAP. We are an internal team focused on managing mobile devices, mobile applications, and developing custom apps for SAP’s 110,000 employees. I believe we have some unique stories, software, tools, and insights to help others in the community considering, or currently undertaking, some of the challenges which surround mobility and its adoption in the enterprise. As an SAP champion, I enjoy sharing and promoting my experiences and knowledge with others through the SAP Community, if you enjoy this post, check out some of my past content.

To me, pervasiveness and versatility are key elements which I greatly respect in the tech scene, and when it comes to new products and services in the consumer technology world. When my teenage kids have friends over … I am one of “those” Dads asking what apps they use and why …. when a new iPhone gets launched, I ask if they heard about it … and in my most recent interaction with our next generation of enterprise workers, I didn’t need to ask, but instead they shared with me how they use ChatGPT to get their homework done, write essays, or answer their teachers’ tricky questions … This is just one of the aspects I use as a temperature gauge, for the impact that some of these technology companies are making on both the current and eventual consumers and customers of their products.

In order to satisfy our current generation of employees needs with tools like ChatGPT, and subsequently be ready for the next, we are integrating innovation into our internal products and process which our employees both expect and want.

Our App Development team is not necessarily new to NLP technologies, in 2016 we introduced Hubot into internal chat solution called Relay and when we deployed it, we were very optimistic about the agent & bot opportunities that were emerging at that time. We then did a proof of concept with some emerging startups like Kore.ai, we ran our own instance of RASA, Nimbus, and due to a company acquisition, also deployed one of SAP’s Conversational tools. In reality, very few scenarios were actually adopted by our users. Our agents were quite static and required a lot of explicitly defined code and actions to be practical or used. And while they were “OK” at interpreting what a user was asking or responding to. One area that we always struggled to process was when users asked questions that were not pre-scripted, or when the spelling was incorrect, or the context was not clearly defined. This often left our bots stumped, and inevitably resulted in a somewhat negative user experience.

So fast forward a few years to where we are today, and I think the reintroduction, or reinvigoration, of NLP technology which started late in 2022 has got our attention again as a potential avenue for new interaction patterns and opportunities and areas for innovation within our existing product lines. Tools like ChatGPT have the unique ability to accurately perform text classification, contextually interpret questions and comments, and precisely predict named entity recognition is what sets it apart from past technologies, and it is revolutionizing multiple aspects of traditional computing.

After multiple brainstorming sessions, user interviews and discussions, one of the apps we felt a LLM could really help transform centered around text generation, in an app called Celebrate. The app encourages and helps spread positivity and kindness by reminding and enabling employees to celebrate special holidays, birthdays and work anniversaries of colleagues, managers, and friends. The app shows the event dates and details, provides push notifications, and can also send a predefined or custom card for the special occasion. We recognized that folks sometimes struggle to find the words to share on a special occasion. Due to this, we implemented ChatGPT using text generation in this app to simplify and provide our users with an option to automatically generate a message with the senders preferred tone and theme.

A screenshot of a cell phone

Description automatically generatedA screenshot of a phone

Description automatically generated

Like the emotional benefit of being connected, appreciated, or loved by people around you, humor has a similar affect, scientifically, it has been shown to reduce stress, enhance your creativity, improve moral and facilitate in team building.

With this in mind, the other app we felt was a good candidate for LLM integration is called Leave Request. It is a small and simple app for requesting vacation time anywhere and at any time. The experience is optimized for a mobile device. The ChatGPT integration lets users create a predefined message based on the dates of the vacation, tone and theme the user wishes to portrait. This message could potentially help users to convey the importance or more humorous side of why they need some time off, this could go a long way to encouraging their manager to make a positive decision.

A screenshot of a phone

Description automatically generatedA screenshot of a phone

Description automatically generated

While both examples are not groundbreaking or particularly relevant to a specific business use case or process at SAP, we are using them to inspire and spark innovation amongst our users, seeding ideas or potentially encouraging employees and teams to reach out to discuss their next big idea or area of opportunity.

Have you implemented ChatGPT or any NLP/LLM into your business processes? Do you think it has been successful or worthwhile? Please share your thoughts into the comments as I am eager to know how others are blending mobility with new innovations to help their companies deliver value and a competitive advantage.