Skip to content

Commit 21b5862

Browse files
author
AWS
committed
Amazon SageMaker Service Update: A new parameter called ExplainerConfig is added to CreateEndpointConfig API to enable SageMaker Clarify online explainability feature.
1 parent d7af3a4 commit 21b5862

File tree

2 files changed

+329
-0
lines changed

2 files changed

+329
-0
lines changed

‎.changes/next-release/feature-AmazonSageMakerService-a2499b3.json

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
{
2+
"type": "feature",
3+
"category": "Amazon SageMaker Service",
4+
"contributor": "",
5+
"description": "A new parameter called ExplainerConfig is added to CreateEndpointConfig API to enable SageMaker Clarify online explainability feature."
6+
}

‎services/sagemaker/src/main/resources/codegen-resources/service-2.json

Lines changed: 323 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5244,6 +5244,307 @@
52445244
},
52455245
"documentation":"<p>The container for the metadata for the ClarifyCheck step. For more information, see the topic on <a href=\"https://docs.aws.amazon.com/sagemaker/latest/dg/build-and-manage-steps.html#step-type-clarify-check\">ClarifyCheck step</a> in the <i>Amazon SageMaker Developer Guide</i>. </p>"
52465246
},
5247+
"ClarifyContentTemplate":{
5248+
"type":"string",
5249+
"max":64,
5250+
"min":1,
5251+
"pattern":".*"
5252+
},
5253+
"ClarifyEnableExplanations":{
5254+
"type":"string",
5255+
"max":64,
5256+
"min":1,
5257+
"pattern":".*"
5258+
},
5259+
"ClarifyExplainerConfig":{
5260+
"type":"structure",
5261+
"required":["ShapConfig"],
5262+
"members":{
5263+
"EnableExplanations":{
5264+
"shape":"ClarifyEnableExplanations",
5265+
"documentation":"<p>A JMESPath boolean expression used to filter which records to explain. Explanations are activated by default. See <a href=\"https://docs.aws.amazon.com/sagemaker-dg/src/AWSIronmanApiDoc/build/server-root/sagemaker/latest/dg/clarify-online-explainability-create-endpoint.html#clarify-online-explainability-create-endpoint-enable\"> <code>EnableExplanations</code> </a>for additional information.</p>"
5266+
},
5267+
"InferenceConfig":{
5268+
"shape":"ClarifyInferenceConfig",
5269+
"documentation":"<p>The inference configuration parameter for the model container.</p>"
5270+
},
5271+
"ShapConfig":{
5272+
"shape":"ClarifyShapConfig",
5273+
"documentation":"<p>The configuration for SHAP analysis.</p>"
5274+
}
5275+
},
5276+
"documentation":"<p>The configuration parameters for the SageMaker Clarify explainer.</p>"
5277+
},
5278+
"ClarifyFeatureHeaders":{
5279+
"type":"list",
5280+
"member":{"shape":"ClarifyHeader"},
5281+
"max":256,
5282+
"min":1
5283+
},
5284+
"ClarifyFeatureType":{
5285+
"type":"string",
5286+
"enum":[
5287+
"numerical",
5288+
"categorical",
5289+
"text"
5290+
]
5291+
},
5292+
"ClarifyFeatureTypes":{
5293+
"type":"list",
5294+
"member":{"shape":"ClarifyFeatureType"},
5295+
"max":256,
5296+
"min":1
5297+
},
5298+
"ClarifyFeaturesAttribute":{
5299+
"type":"string",
5300+
"max":64,
5301+
"min":1,
5302+
"pattern":".*"
5303+
},
5304+
"ClarifyHeader":{
5305+
"type":"string",
5306+
"max":64,
5307+
"min":1,
5308+
"pattern":".*"
5309+
},
5310+
"ClarifyInferenceConfig":{
5311+
"type":"structure",
5312+
"members":{
5313+
"FeaturesAttribute":{
5314+
"shape":"ClarifyFeaturesAttribute",
5315+
"documentation":"<p>Provides the JMESPath expression to extract the features from a model container input in JSON Lines format. For example, if <code>FeaturesAttribute</code> is the JMESPath expression <code>'myfeatures'</code>, it extracts a list of features <code>[1,2,3]</code> from request data <code>'{\"myfeatures\":[1,2,3}'</code>.</p>"
5316+
},
5317+
"ContentTemplate":{
5318+
"shape":"ClarifyContentTemplate",
5319+
"documentation":"<p>A template string used to format a JSON record into an acceptable model container input. For example, a <code>ContentTemplate</code> string <code>'{\"myfeatures\":$features}'</code> will format a list of features <code>[1,2,3]</code> into the record string <code>'{\"myfeatures\":[1,2,3]}'</code>. Required only when the model container input is in JSON Lines format.</p>"
5320+
},
5321+
"MaxRecordCount":{
5322+
"shape":"ClarifyMaxRecordCount",
5323+
"documentation":"<p>The maximum number of records in a request that the model container can process when querying the model container for the predictions of a <a href=\"https://docs.aws.amazon.com/sagemaker/latest/dg/clarify-online-explainability-create-endpoint.html#clarify-online-explainability-create-endpoint-synthetic\">synthetic dataset</a>. A record is a unit of input data that inference can be made on, for example, a single line in CSV data. If <code>MaxRecordCount</code> is <code>1</code>, the model container expects one record per request. A value of 2 or greater means that the model expects batch requests, which can reduce overhead and speed up the inferencing process. If this parameter is not provided, the explainer will tune the record count per request according to the model container's capacity at runtime.</p>"
5324+
},
5325+
"MaxPayloadInMB":{
5326+
"shape":"ClarifyMaxPayloadInMB",
5327+
"documentation":"<p>The maximum payload size (MB) allowed of a request from the explainer to the model container. Defaults to <code>6</code> MB.</p>"
5328+
},
5329+
"ProbabilityIndex":{
5330+
"shape":"ClarifyProbabilityIndex",
5331+
"documentation":"<p>A zero-based index used to extract a probability value (score) or list from model container output in CSV format. If this value is not provided, the entire model container output will be treated as a probability value (score) or list.</p> <p> <b>Example for a single class model:</b> If the model container output consists of a string-formatted prediction label followed by its probability: <code>'1,0.6'</code>, set <code>ProbabilityIndex</code> to <code>1</code> to select the probability value <code>0.6</code>.</p> <p> <b>Example for a multiclass model:</b> If the model container output consists of a string-formatted prediction label followed by its probability: <code>'\"[\\'cat\\',\\'dog\\',\\'fish\\']\",\"[0.1,0.6,0.3]\"'</code>, set <code>ProbabilityIndex</code> to <code>1</code> to select the probability values <code>[0.1,0.6,0.3]</code>.</p>"
5332+
},
5333+
"LabelIndex":{
5334+
"shape":"ClarifyLabelIndex",
5335+
"documentation":"<p>A zero-based index used to extract a label header or list of label headers from model container output in CSV format.</p> <p> <b>Example for a multiclass model:</b> If the model container output consists of label headers followed by probabilities: <code>'\"[\\'cat\\',\\'dog\\',\\'fish\\']\",\"[0.1,0.6,0.3]\"'</code>, set <code>LabelIndex</code> to <code>0</code> to select the label headers <code>['cat','dog','fish']</code>.</p>"
5336+
},
5337+
"ProbabilityAttribute":{
5338+
"shape":"ClarifyProbabilityAttribute",
5339+
"documentation":"<p>A JMESPath expression used to extract the probability (or score) from the model container output if the model container is in JSON Lines format.</p> <p> <b>Example</b>: If the model container output of a single request is <code>'{\"predicted_label\":1,\"probability\":0.6}'</code>, then set <code>ProbabilityAttribute</code> to <code>'probability'</code>.</p>"
5340+
},
5341+
"LabelAttribute":{
5342+
"shape":"ClarifyLabelAttribute",
5343+
"documentation":"<p>A JMESPath expression used to locate the list of label headers in the model container output.</p> <p> <b>Example</b>: If the model container output of a batch request is <code>'{\"labels\":[\"cat\",\"dog\",\"fish\"],\"probability\":[0.6,0.3,0.1]}'</code>, then set <code>LabelAttribute</code> to <code>'labels'</code> to extract the list of label headers <code>[\"cat\",\"dog\",\"fish\"]</code> </p>"
5344+
},
5345+
"LabelHeaders":{
5346+
"shape":"ClarifyLabelHeaders",
5347+
"documentation":"<p>For multiclass classification problems, the label headers are the names of the classes. Otherwise, the label header is the name of the predicted label. These are used to help readability for the output of the <code>InvokeEndpoint</code> API. See the <a href=\"https://docs.aws.amazon.com/sagemaker/latest/dg/clarify-online-explainability-invoke-endpoint.html#clarify-online-explainability-response\">response</a> section under <b>Invoke the endpoint</b> in the Developer Guide for more information. If there are no label headers in the model container output, provide them manually using this parameter.</p>"
5348+
},
5349+
"FeatureHeaders":{
5350+
"shape":"ClarifyFeatureHeaders",
5351+
"documentation":"<p>The names of the features. If provided, these are included in the endpoint response payload to help readability of the <code>InvokeEndpoint</code> output. See the <a href=\"https://docs.aws.amazon.com/sagemaker/latest/dg/clarify-online-explainability-invoke-endpoint.html#clarify-online-explainability-response\">Response</a> section under <b>Invoke the endpoint</b> in the Developer Guide for more information.</p>"
5352+
},
5353+
"FeatureTypes":{
5354+
"shape":"ClarifyFeatureTypes",
5355+
"documentation":"<p>A list of data types of the features (optional). Applicable only to NLP explainability. If provided, <code>FeatureTypes</code> must have at least one <code>'text'</code> string (for example, <code>['text']</code>). If <code>FeatureTypes</code> is not provided, the explainer infers the feature types based on the baseline data. The feature types are included in the endpoint response payload. For additional information see the <a href=\"https://docs.aws.amazon.com/sagemaker/latest/dg/clarify-online-explainability-invoke-endpoint.html#clarify-online-explainability-response\">response</a> section under <b>Invoke the endpoint</b> in the Developer Guide for more information.</p>"
5356+
}
5357+
},
5358+
"documentation":"<p>The inference configuration parameter for the model container.</p>"
5359+
},
5360+
"ClarifyLabelAttribute":{
5361+
"type":"string",
5362+
"max":64,
5363+
"min":1,
5364+
"pattern":".*"
5365+
},
5366+
"ClarifyLabelHeaders":{
5367+
"type":"list",
5368+
"member":{"shape":"ClarifyHeader"},
5369+
"max":16,
5370+
"min":1
5371+
},
5372+
"ClarifyLabelIndex":{
5373+
"type":"integer",
5374+
"min":0
5375+
},
5376+
"ClarifyMaxPayloadInMB":{
5377+
"type":"integer",
5378+
"max":25,
5379+
"min":1
5380+
},
5381+
"ClarifyMaxRecordCount":{
5382+
"type":"integer",
5383+
"min":1
5384+
},
5385+
"ClarifyMimeType":{
5386+
"type":"string",
5387+
"max":255,
5388+
"pattern":"^[a-zA-Z0-9](-*[a-zA-Z0-9])*\\/[a-zA-Z0-9](-*[a-zA-Z0-9+.])*"
5389+
},
5390+
"ClarifyProbabilityAttribute":{
5391+
"type":"string",
5392+
"max":64,
5393+
"min":1,
5394+
"pattern":".*"
5395+
},
5396+
"ClarifyProbabilityIndex":{
5397+
"type":"integer",
5398+
"min":0
5399+
},
5400+
"ClarifyShapBaseline":{
5401+
"type":"string",
5402+
"max":4096,
5403+
"min":1,
5404+
"pattern":"[\\s\\S]+"
5405+
},
5406+
"ClarifyShapBaselineConfig":{
5407+
"type":"structure",
5408+
"members":{
5409+
"MimeType":{
5410+
"shape":"ClarifyMimeType",
5411+
"documentation":"<p>The MIME type of the baseline data. Choose from <code>'text/csv'</code> or <code>'application/jsonlines'</code>. Defaults to <code>'text/csv'</code>.</p>"
5412+
},
5413+
"ShapBaseline":{
5414+
"shape":"ClarifyShapBaseline",
5415+
"documentation":"<p>The inline SHAP baseline data in string format. <code>ShapBaseline</code> can have one or multiple records to be used as the baseline dataset. The format of the SHAP baseline file should be the same format as the training dataset. For example, if the training dataset is in CSV format and each record contains four features, and all features are numerical, then the format of the baseline data should also share these characteristics. For natural language processing (NLP) of text columns, the baseline value should be the value used to replace the unit of text specified by the <code>Granularity</code> of the <code>TextConfig</code> parameter. The size limit for <code>ShapBasline</code> is 4 KB. Use the <code>ShapBaselineUri</code> parameter if you want to provide more than 4 KB of baseline data.</p>"
5416+
},
5417+
"ShapBaselineUri":{
5418+
"shape":"Url",
5419+
"documentation":"<p>The uniform resource identifier (URI) of the S3 bucket where the SHAP baseline file is stored. The format of the SHAP baseline file should be the same format as the format of the training dataset. For example, if the training dataset is in CSV format, and each record in the training dataset has four features, and all features are numerical, then the baseline file should also have this same format. Each record should contain only the features. If you are using a virtual private cloud (VPC), the <code>ShapBaselineUri</code> should be accessible to the VPC. For more information about setting up endpoints with Amazon Virtual Private Cloud, see <a href=\"https://docs.aws.amazon.com/sagemaker/latest/dg/infrastructure-give-access.html\">Give SageMaker access to Resources in your Amazon Virtual Private Cloud</a>.</p>"
5420+
}
5421+
},
5422+
"documentation":"<p>The configuration for the <a href=\"https://docs.aws.amazon.com/sagemaker/latest/dg/clarify-feature-attribute-shap-baselines.html\">SHAP baseline</a> (also called the background or reference dataset) of the Kernal SHAP algorithm.</p> <note> <ul> <li> <p>The number of records in the baseline data determines the size of the synthetic dataset, which has an impact on latency of explainability requests. For more information, see the <b>Synthetic data</b> of <a href=\"https://docs.aws.amazon.com/sagemaker/latest/dg/clarify-online-explainability-create-endpoint.html\">Configure and create an endpoint</a>.</p> </li> <li> <p> <code>ShapBaseline</code> and <code>ShapBaselineUri</code> are mutually exclusive parameters. One or the either is required to configure a SHAP baseline. </p> </li> </ul> </note>"
5423+
},
5424+
"ClarifyShapConfig":{
5425+
"type":"structure",
5426+
"required":["ShapBaselineConfig"],
5427+
"members":{
5428+
"ShapBaselineConfig":{
5429+
"shape":"ClarifyShapBaselineConfig",
5430+
"documentation":"<p>The configuration for the SHAP baseline of the Kernal SHAP algorithm.</p>"
5431+
},
5432+
"NumberOfSamples":{
5433+
"shape":"ClarifyShapNumberOfSamples",
5434+
"documentation":"<p>The number of samples to be used for analysis by the Kernal SHAP algorithm. </p> <note> <p>The number of samples determines the size of the synthetic dataset, which has an impact on latency of explainability requests. For more information, see the <b>Synthetic data</b> of <a href=\"https://docs.aws.amazon.com/sagemaker/latest/dg/clarify-online-explainability-create-endpoint.html\">Configure and create an endpoint</a>.</p> </note>"
5435+
},
5436+
"UseLogit":{
5437+
"shape":"ClarifyShapUseLogit",
5438+
"documentation":"<p>A Boolean toggle to indicate if you want to use the logit function (true) or log-odds units (false) for model predictions. Defaults to false.</p>"
5439+
},
5440+
"Seed":{
5441+
"shape":"ClarifyShapSeed",
5442+
"documentation":"<p>The starting value used to initialize the random number generator in the explainer. Provide a value for this parameter to obtain a deterministic SHAP result.</p>"
5443+
},
5444+
"TextConfig":{
5445+
"shape":"ClarifyTextConfig",
5446+
"documentation":"<p>A parameter that indicates if text features are treated as text and explanations are provided for individual units of text. Required for natural language processing (NLP) explainability only.</p>"
5447+
}
5448+
},
5449+
"documentation":"<p>The configuration for SHAP analysis using SageMaker Clarify Explainer.</p>"
5450+
},
5451+
"ClarifyShapNumberOfSamples":{
5452+
"type":"integer",
5453+
"min":1
5454+
},
5455+
"ClarifyShapSeed":{"type":"integer"},
5456+
"ClarifyShapUseLogit":{"type":"boolean"},
5457+
"ClarifyTextConfig":{
5458+
"type":"structure",
5459+
"required":[
5460+
"Language",
5461+
"Granularity"
5462+
],
5463+
"members":{
5464+
"Language":{
5465+
"shape":"ClarifyTextLanguage",
5466+
"documentation":"<p>Specifies the language of the text features in <a href=\" https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes\">ISO 639-1</a> or <a href=\"https://en.wikipedia.org/wiki/ISO_639-3\">ISO 639-3</a> code of a supported language. </p> <note> <p>For a mix of multiple languages, use code <code>'xx'</code>.</p> </note>"
5467+
},
5468+
"Granularity":{
5469+
"shape":"ClarifyTextGranularity",
5470+
"documentation":"<p>The unit of granularity for the analysis of text features. For example, if the unit is <code>'token'</code>, then each token (like a word in English) of the text is treated as a feature. SHAP values are computed for each unit/feature.</p>"
5471+
}
5472+
},
5473+
"documentation":"<p>A parameter used to configure the SageMaker Clarify explainer to treat text features as text so that explanations are provided for individual units of text. Required only for natural language processing (NLP) explainability. </p>"
5474+
},
5475+
"ClarifyTextGranularity":{
5476+
"type":"string",
5477+
"enum":[
5478+
"token",
5479+
"sentence",
5480+
"paragraph"
5481+
]
5482+
},
5483+
"ClarifyTextLanguage":{
5484+
"type":"string",
5485+
"enum":[
5486+
"af",
5487+
"sq",
5488+
"ar",
5489+
"hy",
5490+
"eu",
5491+
"bn",
5492+
"bg",
5493+
"ca",
5494+
"zh",
5495+
"hr",
5496+
"cs",
5497+
"da",
5498+
"nl",
5499+
"en",
5500+
"et",
5501+
"fi",
5502+
"fr",
5503+
"de",
5504+
"el",
5505+
"gu",
5506+
"he",
5507+
"hi",
5508+
"hu",
5509+
"is",
5510+
"id",
5511+
"ga",
5512+
"it",
5513+
"kn",
5514+
"ky",
5515+
"lv",
5516+
"lt",
5517+
"lb",
5518+
"mk",
5519+
"ml",
5520+
"mr",
5521+
"ne",
5522+
"nb",
5523+
"fa",
5524+
"pl",
5525+
"pt",
5526+
"ro",
5527+
"ru",
5528+
"sa",
5529+
"sr",
5530+
"tn",
5531+
"si",
5532+
"sk",
5533+
"sl",
5534+
"es",
5535+
"sv",
5536+
"tl",
5537+
"ta",
5538+
"tt",
5539+
"te",
5540+
"tr",
5541+
"uk",
5542+
"ur",
5543+
"yo",
5544+
"lij",
5545+
"xx"
5546+
]
5547+
},
52475548
"ClientId":{
52485549
"type":"string",
52495550
"max":1024,
@@ -6452,6 +6753,10 @@
64526753
"AsyncInferenceConfig":{
64536754
"shape":"AsyncInferenceConfig",
64546755
"documentation":"<p>Specifies configuration for how an endpoint performs asynchronous inference. This is a required field in order for your Endpoint to be invoked using <a href=\"https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_runtime_InvokeEndpointAsync.html\">InvokeEndpointAsync</a>.</p>"
6756+
},
6757+
"ExplainerConfig":{
6758+
"shape":"ExplainerConfig",
6759+
"documentation":"<p>A member of <code>CreateEndpointConfig</code> that enables explainers.</p>"
64556760
}
64566761
}
64576762
},
@@ -10147,6 +10452,10 @@
1014710452
"AsyncInferenceConfig":{
1014810453
"shape":"AsyncInferenceConfig",
1014910454
"documentation":"<p>Returns the description of an endpoint configuration created using the <a href=\"https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateEndpointConfig.html\"> <code>CreateEndpointConfig</code> </a> API.</p>"
10455+
},
10456+
"ExplainerConfig":{
10457+
"shape":"ExplainerConfig",
10458+
"documentation":"<p>The configuration parameters for an explainer.</p>"
1015010459
}
1015110460
}
1015210461
},
@@ -10215,6 +10524,10 @@
1021510524
"PendingDeploymentSummary":{
1021610525
"shape":"PendingDeploymentSummary",
1021710526
"documentation":"<p>Returns the summary of an in-progress deployment. This field is only returned when the endpoint is creating or updating with a new endpoint configuration.</p>"
10527+
},
10528+
"ExplainerConfig":{
10529+
"shape":"ExplainerConfig",
10530+
"documentation":"<p>The configuration parameters for an explainer.</p>"
1021810531
}
1021910532
}
1022010533
},
@@ -13906,6 +14219,16 @@
1390614219
"type":"string",
1390714220
"min":1
1390814221
},
14222+
"ExplainerConfig":{
14223+
"type":"structure",
14224+
"members":{
14225+
"ClarifyExplainerConfig":{
14226+
"shape":"ClarifyExplainerConfig",
14227+
"documentation":"<p>A member of <code>ExplainerConfig</code> that contains configuration parameters for the SageMaker Clarify explainer.</p>"
14228+
}
14229+
},
14230+
"documentation":"<p>A parameter to activate explainers.</p>"
14231+
},
1390914232
"FailStepMetadata":{
1391014233
"type":"structure",
1391114234
"members":{

0 commit comments

Comments
 (0)