In a previous post, we built a custom model hosted in AWS SageMaker to generate Customer Success Scorecards (CSS). In this post, we’ll greatly improve the impact of our project by using Agentforce to build an agent that can tap into all of the previously built infrastructure to answer user questions about the model and its results.
This agent provides a unique interface that allows users to not only have “conversations” with their data, but it can also perform experimentation actions that would normally required a dedicated analytics team. For example, a user could ask questions like:
- What are the main impacts to the score for this account?
- What happens if an account activates ten new licenses?
- If an account stops using workflows, what will happen to the score?
- What is the most impactful change I can make?
The patterns used in this post will show you how to connect an agent to existing data in Salesforce CRM and Data Cloud, and trigger calls to Einstein Studio’s Bring Your Own Model capability.
Note: Make sure that you have a working version of the Customer Success Scorecard that we built together in the previous post as we will be reusing some components here, along with a CSS Recommendations topic and a couple of flows to tie it all together.
Architecture
Before diving into the technical instructions, let’s go over the big picture. Below we show the end-to-end architecture we’ll build together (everything in gray was built in the previous post, and new features are outlined in black).
- First, we need to construct the two flows: one to load account information and the other to run experiments
- Second, we need to define our agent and provide instructions, so it knows the topic and what actions to call and when
By leveraging Data Cloud, we can pull information from across Salesforce, including the data that is written by the CSS model and all the original metrics. This data will ultimately be used by the agent to make recommendations and run experiments.
Step 1: Build the data and experiment flows
Before we can work on our agent, we need to build two flows: “CSS Recommendations Data” and “CSS Recommendations Experiment.” The first flow will aggregate all the data relevant to an account’s CSS score and the latter will be used to trigger the SageMaker endpoint we built in the previous blog post with modified values.
CSS Recommendations Data flow
For the CSS Recommendations Data flow, we’ll need to accept an account identifier as the input, gather all the relevant information from Data Cloud, and finally return that information as a string.
- Navigate to Setup > Flows
- Click New Flow > Start from Scratch > Autolaunched Flow (No Trigger) > Create
- Click New Resource to create an input variable. Add “AccountID” as the API name and select “Text” as the data type. Make sure that “Available for input” is checked! This will allow the agent to pass the account information to the flow. See the screenshot below.
- Next, we’ll need four Get Records elements, one to pull in each component of the Account’s information.
The screenshot below shows the content for the Get Account Details element. Set the data source to Data Cloud Object with object type AccountMini filtered based on the input variable “AccountID.” See the screenshot below.
Add one for each of the remaining DMOs: “usage_metrics”, “product_metrics”, and “CSS_Mini”. When you are done, your flow should look like this:
- Now we need to return all of this information to the agent, so let’s create a text template resource named “Account_details” with the following content. This will map all data retrieved to a single string for the agent.
{
"account_id": "{!AccountID}",
"inputs": {
"account_age": {!Get_Account_Details.Age_c__c},
"account_size": {!Get_Account_Details.Size_c__c},
"activation_rate": {!Get_Product_Metrics.activation_rate__c},
"penetration_rate": {!Get_Product_Metrics.penetration_rate__c},
"engagement_rate": {!Get_Product_Metrics.engagement_rate__c},
"utilization_rate": {!Get_Product_Metrics.utilization_rate__c},
"bots": {!Get_Usage_Metrics.bots__c},
"workflows": {!Get_Usage_Metrics.workflows__c},
"email2case": {!Get_Usage_Metrics.email2case__c},
"lightning_utilization": {!Get_Usage_Metrics.lightning_utilization__c},
"macro": {!Get_Usage_Metrics.macro__c},
"workflowrules": {!Get_Usage_Metrics.workflowrules__c},
"quicktext": {!Get_Usage_Metrics.quicktext__c},
"web2case": {!Get_Usage_Metrics.web2case__c},
"ltnflows": {!Get_Usage_Metrics.ltnflows__c}
},
"scores": {
"consumption_engagement_rate_score": {!Get_CSS_Score.consumption_engagement_rate_score__c},
"consumption_activation_rate_score": {!Get_CSS_Score.consumption_activation_rate_score__c},
"consumption_penetration_rate_score": {!Get_CSS_Score.consumption_penetration_rate_score__c},
"consumption_utilization_rate_score": {!Get_CSS_Score.consumption_utilization_rate_score__c},
"auto_bots_score": {!Get_CSS_Score.auto_bots_score__c},
"auto_email2case_score": {!Get_CSS_Score.auto_email2case_score__c},
"auto_ltnflows_score": {!Get_CSS_Score.auto_ltnflows_score__c},
"auto_macro_score": {!Get_CSS_Score.auto_macro_score__c},
"auto_quicktext_score": {!Get_CSS_Score.auto_quicktext_score__c},
"auto_web2case_score": {!Get_CSS_Score.auto_web2case_score__c},
"auto_workflowrules_score": {!Get_CSS_Score.auto_workflowrules_score__c},
"auto_workflows_score": {!Get_CSS_Score.auto_workflows_score__c},
"lightning_utilization_score": {!Get_CSS_Score.lightning_utilization_score__c},
"global_score": {!Get_CSS_Score.global_score__c},
"auto_component_score": {!Get_CSS_Score.auto_component_score__c},
"consumption_component_score": {!Get_CSS_Score.consumption_component_score__c},
"lightning_component_score": {!Get_CSS_Score.lightning_component_score__c}
}
}
- We also need an output variable. Add “Output” as the API name and select “Text” as the data type. Be sure to check “Available for output”! See the screenshot below.
- As the final step in our flow, we need to assign the text template to the output variable using the Assignment element. Add “Set Output” in the Label field, and set the Output variable equal to the Account_Details text field. See the screenshot below.
- Finally, we can save the flow, give it a name, and activate it.
At this point you can use the flow debugging tool to test the flow and make sure everything works. Provide an account ID from the AccountMini object and verify that the flow executes correctly and returns all of the needed information.
CSS Recommendations Experiment flow
Moving on to our second flow, CSS Recommendations Experiment, we will need to accept all the parameters of the CSS model as inputs, run the model in Einstein Studio, and return the results as a string.
- Navigate to Setup > Flows
- Click New Flow > Start from Scratch > Autolaunched Flow (No Trigger) > Create
- Click New Resource to create an input variable for each item in the following table. Make sure that “Available for input” is checked!
acct_age acct_size activation_rate bots email2case engagement_rate lightning_utilization ltnflows macro penetration_rate quicktext utilization_rate web2case workflowrules workflows
- Now add a new Action element and search for the name of your CSS Mini model in Einstein studio. Then map each of the input variables that we just added to the model inputs.
In the screenshot below, “Label” is set to “Run Experiment” and each input value is mapped to the corresponding flow variable.
- Similar to the first flow, we need to map the result of the flow to an output variable. So, create a new text template and add an assignment to an output variable. The text template should have the following content.
{
"consumption_engagement_rate_score": {!Run_Experiment.param_consumption_engagement_rate_score},
"consumption_activation_rate_score": {!Run_Experiment.param_consumption_activation_rate_score},
"consumption_penetration_rate_score": {!Run_Experiment.param_consumption_penetration_rate_score},
"consumption_utilization_rate_score": {!Run_Experiment.param_consumption_utilization_rate_score},
"auto_bots_score": {!Run_Experiment.param_auto_bots_score},
"auto_email2case_score": {!Run_Experiment.param_auto_email2case_score},
"auto_ltnflows_score": {!Run_Experiment.param_auto_ltnflows_score},
"auto_macro_score": {!Run_Experiment.param_auto_macro_score},
"auto_quicktext_score": {!Run_Experiment.param_auto_quicktext_score},
"auto_web2case_score": {!Run_Experiment.param_auto_web2case_score},
"auto_workflowrules_score": {!Run_Experiment.param_auto_workflowrules_score},
"auto_workflows_score": {!Run_Experiment.param_auto_workflows_score},
"lightning_utilization_score": {!Run_Experiment.param_lightning_utilization_score},
"global_score": {!Run_Experiment.param_global_score},
"auto_component_score": {!Run_Experiment.param_auto_component_score},
"consumption_component_score": {!Run_Experiment.param_consumption_component_score},
"lightning_component_score": {!Run_Experiment.param_lightning_component_score}
}
- Finally, we can save the flow and activate it.
Again, make sure that you test the flow. The inputs don’t matter much here, we just want to test that the connection to our custom model in Einstein studio works properly.
Step 2: Create the CSS Recommendations agent
With the flows configured and tested, we can now build our agent! If you have not done so already, you will first need to enable Agentforce.
- Navigate to Setup > Einstein > Einstein Generative AI > Agent Studio > Agents
- Click on New Agent. Enter “CSS Recs” as the name, “This agent provides recommendations based on CSS Scores” as the description, and for the custom agent user, pick a user who has the rights to invoke the two flows we created above.
Once created, your agent details should look like this:
- Click Open in Builder
- Under Topics, Click New > New Topic
- Name the topic “Customer Recommendations” and enter the following information.
| Field | Description | Value |
| Topic Label | This is the name of our topic. | Customer Recommendations |
| Classification Description | This is used to determine when to trigger this topic. | This topic addresses all user questions relating to the Customer Success Score (CSS). |
| Scope | These are general instructions for the agent. | Your job is to assist users in navigating the different metrics that make up the Customer Success Score and give them advice on how to improve their scores. |
| Instructions 1 | This instruction block tells the agent to run experiments and sumarize the results. | When asked how to improve a score, you should run multiple (more than one, but no more than three) “experiments.” Follow these guidelines: * Your experiments should take low scoring metrics and change them by a reasonable amount. Don’t just say to make it the maximum. Use the business context below to understand how much you can change the input. * Each experiment should start from the original metrics, not from other experiments. * Change no more than four individual metrics in each run. * Do not run more experiments than you need to answer the prompt. If a user asks for specific changes or the most impactful change, you only need to run one experiment. * Format your response as markdown. * Summarize the impact of these changes in a clear and user-friendly manner, so that it can be shared with the customer. * Make sure that you include the metric values and global scores in your answer, so the customer understands the impact. * Do not give any additional information, just state the results. * Your response must contain a list of experiments with a summary and state why you changed the metrics. Here is an example format: ”’ Description of what was run Experiment 1: summary and why Experiment 2: summary and why Experiment …: summary and why ”’ |
| Instructions 2 | This block provides a simple summary of CSS, taken from the Salesforce website. | CSS business context: “The Customer Success Score is a simple and comprehensive measure of your success with Salesforce and is accessible to Signature Success Plan customers through Salesforce Help. Get insight into your use of Salesforce products and features, the health of your implementations, and your engagements with learning resources. Both you and Salesforce are accountable for reaching the common goal of your success. The Customer Success Score establishes shared metrics and deepens partnerships. With this feature, you can access insights, track your organization’s progress, and see personalized recommendations to get the most out of your investment with the full potential of CRM + Data + AI + Trust. With the Customer Success Score, you can: Gain insights into your organization’s performance through a self-service dashboard. Track your progress and make data-driven decisions using historical trends and updates from your Customer Success Score. Take action and optimize your implementation with personalized and curated recommendations. Watch the Customer Success Score Overview video to learn more.” |
| Instructions 3 | This block provides a description of each parameter to our CSS Mini model. The agent will use this to determine which parameters to modify when running experiments. | The following table lists each metric that customers are scored on and their descriptions: activation_rate: licenses activated / licenses sold. utilization_rate: MAU / licenses activated. engagement_rate: DAU / MAU. email2case: Automated process to turn incoming email requests into new cases. web2case: Automated process to turn website form requests into new cases. macro: A set of instructions that tells the system how to automatically complete a repetitive task. When a user runs a macro, the system performs each instruction. Macros help your team save time and add consistency. quicktext: Enables users to insert predefined messages into customer conversations and records. Quick text categories can be customized to make a message’s purpose clear. workflows: A container or business logic engine which automates certain actions based on particular criteria. If the criteria are met, the actions get executed. When they are not met, records will get saved but no action will get executed. workflowrules: A workflow rule is the main container for a set of workflow instructions. These instructions can always be summed up in an if/then statement. ltnflow: A flow is the part of Salesforce Flow that collects data and performs actions in your Salesforce org or an external system. Salesforce Flow provides two types of flows: screen flows and autolaunched flows. bots: Automated chat conversation service built on Salesforce app to interact with users and answer questions. lightning_utilization: Percent of MAU on Lightning (Lighting MAU / Total MAU). |
- Once the topic is created, select it from the left panel and navigate to This Topics’s Actions. Here we will link our flows that the agent will use to work with Data Cloud to get recommendations for the user.
- Select New > + Add Action. Enter the following details: “Flow” as the reference action type, “CSS Recs Flow” as the reference action, and “CSS Recs Flow” as the agent action label. Click Next. See the screenshot below.
- Provide the following instructions for the topic and input/output variables:
- Agent action instructions: This should be “Use this flow to gather information from Data Cloud about an account’s Customer Success Score (CSS) and metrics.“
- account_id: This should be “Account Name”
- Output: This should be “JSON Data containing customer information”
See the screenshot below.
- Save the action
- Add the second flow using the same process with the following information:
- Agent action instructions: This should be “Use this action to run a CSS experiment.”
- For each of the inputs: Just enter the name of the variable, since we are being consistent about these, Agentforce will be able to handle the mapping
- Output: This should be “JSON Data containing experiment result scores”
- Save the action
Now, we can finally test the agent. Let’s start simple and ask “Tell me about the CSS Score for account adf266bd2b1.” This will trigger the first flow and return all the relevant information for that account.
This screenshot shows a conversation between a human and the agent. The agent responds with the current scores as a table.
From here, we can ask a follow up question like: “What can this customer do to improve their score?” This will trigger our second flow multiple times to run a few experiments, and the agent will then summarize the results.
This screenshot shows a continuation of the previous conversation between a human and the agent. The agent responds with the details of the three experiments it ran, including what values were modified and the resulting scores.
Conclusion
Congratulations! You built and deployed your own mini Customer Success Scorecard agent! You can now interact with this agent to gain insight and run experiments with your custom CSS model. We hope that you found this exercise valuable, and that you can take these learning and successfully apply them to your own systems.
Resources
- Salesforce Developers Blog: Build Your Own Customer Success Scorecard with Amazon SageMaker and Data Cloud
- Salesforce Developers Blog: Bring Your Own AI Models to Data Cloud
About the authors
Nathan Calandra is a Lead Machine Learning Engineer at Salesforce building infrastructure and platforms for data scientists. He has ten years of experience in software engineering with a focus on all things data. Connect with him on LinkedIn.
Jaswanth Billakanti is a Lead Machine Learning Engineer at Salesforce with over eight years of experience in managing and building infrastructure and platforms, including a strong emphasis on automation. Recently, he has been focusing extensively on data platforms, collaborating with data scientists to deliver efficient and scalable solutions. Connect with him on LinkedIn.
The post Extend Your Customer Success Scorecard with Agentforce appeared first on Salesforce Developers Blog.
