Amazon Forecast Service Update: Predictor creation now supports selecting an accuracy metric to optimize in AutoML and hyperparameter optimization. This release adds additional accuracy metrics for predictors - AverageWeightedQuantileLoss, MAPE and MASE.

Author: AWS

.changes/next-release/feature-AmazonForecastService-ac6754a.json

@@ -0,0 +1,6 @@
+{
+    "type": "feature",
+    "category": "Amazon Forecast Service",
+    "contributor": "",
+    "description": "Predictor creation now supports selecting an accuracy metric to optimize in AutoML and hyperparameter optimization. This release adds additional accuracy metrics for predictors - AverageWeightedQuantileLoss, MAPE and MASE."
+}

services/forecast/src/main/resources/codegen-resources/service-2.json

@@ -896,7 +896,7 @@
         },
         "AutoMLOverrideStrategy":{
           "shape":"AutoMLOverrideStrategy",
-          "documentation":"<p>Used to overide the default AutoML strategy, which is to optimize predictor accuracy. To apply an AutoML strategy that minimizes training time, use <code>LatencyOptimized</code>.</p> <p>This parameter is only valid for predictors trained using AutoML.</p>"
+          "documentation":"<note> <p> The <code>LatencyOptimized</code> AutoML override strategy is only available in private beta. Contact AWS Support or your account manager to learn more about access privileges. </p> </note> <p>Used to overide the default AutoML strategy, which is to optimize predictor accuracy. To apply an AutoML strategy that minimizes training time, use <code>LatencyOptimized</code>.</p> <p>This parameter is only valid for predictors trained using AutoML.</p>"
         },
         "PerformHPO":{
           "shape":"Boolean",
@@ -929,6 +929,10 @@
         "Tags":{
           "shape":"Tags",
           "documentation":"<p>The optional metadata that you apply to the predictor to help you categorize and organize them. Each tag consists of a key and an optional value, both of which you define.</p> <p>The following basic restrictions apply to tags:</p> <ul> <li> <p>Maximum number of tags per resource - 50.</p> </li> <li> <p>For each resource, each tag key must be unique, and each tag key can have only one value.</p> </li> <li> <p>Maximum key length - 128 Unicode characters in UTF-8.</p> </li> <li> <p>Maximum value length - 256 Unicode characters in UTF-8.</p> </li> <li> <p>If your tagging schema is used across multiple services and resources, remember that other services may have restrictions on allowed characters. Generally allowed characters are: letters, numbers, and spaces representable in UTF-8, and the following characters: + - = . _ : / @.</p> </li> <li> <p>Tag keys and values are case sensitive.</p> </li> <li> <p>Do not use <code>aws:</code>, <code>AWS:</code>, or any upper or lowercase combination of such as a prefix for keys as it is reserved for AWS use. You cannot edit or delete tag keys with this prefix. Values can have this prefix. If a tag value has <code>aws</code> as its prefix but the key does not, then Forecast considers it to be a user tag and will count against the limit of 50 tags. Tags with only the key prefix of <code>aws</code> do not count against your tags per resource limit.</p> </li> </ul>"
+        },
+        "OptimizationMetric":{
+          "shape":"OptimizationMetric",
+          "documentation":"<p>The accuracy metric used to optimize the predictor.</p>"
         }
       }
     },
@@ -1507,7 +1511,7 @@
         },
         "AutoMLOverrideStrategy":{
           "shape":"AutoMLOverrideStrategy",
-          "documentation":"<p>The AutoML strategy used to train the predictor. Unless <code>LatencyOptimized</code> is specified, the AutoML strategy optimizes predictor accuracy.</p> <p>This parameter is only valid for predictors trained using AutoML.</p>"
+          "documentation":"<note> <p> The <code>LatencyOptimized</code> AutoML override strategy is only available in private beta. Contact AWS Support or your account manager to learn more about access privileges. </p> </note> <p>The AutoML strategy used to train the predictor. Unless <code>LatencyOptimized</code> is specified, the AutoML strategy optimizes predictor accuracy.</p> <p>This parameter is only valid for predictors trained using AutoML.</p>"
         },
         "PerformHPO":{
           "shape":"Boolean",
@@ -1568,6 +1572,10 @@
         "LastModificationTime":{
           "shape":"Timestamp",
           "documentation":"<p>The last time the resource was modified. The timestamp depends on the status of the job:</p> <ul> <li> <p> <code>CREATE_PENDING</code> - The <code>CreationTime</code>.</p> </li> <li> <p> <code>CREATE_IN_PROGRESS</code> - The current timestamp.</p> </li> <li> <p> <code>CREATE_STOPPING</code> - The current timestamp.</p> </li> <li> <p> <code>CREATE_STOPPED</code> - When the job stopped.</p> </li> <li> <p> <code>ACTIVE</code> or <code>CREATE_FAILED</code> - When the job finished or failed.</p> </li> </ul>"
+        },
+        "OptimizationMetric":{
+          "shape":"OptimizationMetric",
+          "documentation":"<p>The accuracy metric used to optimize the predictor.</p>"
         }
       }
     },
@@ -1608,7 +1616,7 @@
       "members":{
         "ForecastType":{
           "shape":"ForecastType",
-          "documentation":"<p> The Forecast type used to compute WAPE and RMSE. </p>"
+          "documentation":"<p> The Forecast type used to compute WAPE, MAPE, MASE, and RMSE. </p>"
         },
         "WAPE":{
           "shape":"Double",
@@ -1617,6 +1625,14 @@
         "RMSE":{
           "shape":"Double",
           "documentation":"<p> The root-mean-square error (RMSE). </p>"
+        },
+        "MASE":{
+          "shape":"Double",
+          "documentation":"<p>The Mean Absolute Scaled Error (MASE)</p>"
+        },
+        "MAPE":{
+          "shape":"Double",
+          "documentation":"<p>The Mean Absolute Percentage Error (MAPE)</p>"
         }
       },
       "documentation":"<p> Provides detailed error metrics to evaluate the performance of a predictor. This object is part of the <a>Metrics</a> object. </p>"
@@ -1895,7 +1911,11 @@
         },
         "AutoMLOverrideStrategy":{
           "shape":"AutoMLOverrideStrategy",
-          "documentation":"<p>The AutoML strategy used to train the predictor. Unless <code>LatencyOptimized</code> is specified, the AutoML strategy optimizes predictor accuracy.</p> <p>This parameter is only valid for predictors trained using AutoML.</p>"
+          "documentation":"<note> <p> The <code>LatencyOptimized</code> AutoML override strategy is only available in private beta. Contact AWS Support or your account manager to learn more about access privileges. </p> </note> <p>The AutoML strategy used to train the predictor. Unless <code>LatencyOptimized</code> is specified, the AutoML strategy optimizes predictor accuracy.</p> <p>This parameter is only valid for predictors trained using AutoML.</p>"
+        },
+        "OptimizationMetric":{
+          "shape":"OptimizationMetric",
+          "documentation":"<p>The accuracy metric used to optimize the predictor.</p>"
         }
       }
     },
@@ -2230,7 +2250,11 @@
         },
         "ErrorMetrics":{
           "shape":"ErrorMetrics",
-          "documentation":"<p> Provides detailed error metrics on forecast type, root-mean square-error (RMSE), and weighted average percentage error (WAPE). </p>"
+          "documentation":"<p> Provides detailed error metrics for each forecast type. Metrics include root-mean square-error (RMSE), mean absolute percentage error (MAPE), mean absolute scaled error (MASE), and weighted average percentage error (WAPE). </p>"
+        },
+        "AverageWeightedQuantileLoss":{
+          "shape":"Double",
+          "documentation":"<p>The average value of all weighted quantile losses.</p>"
         }
       },
       "documentation":"<p>Provides metrics that are used to evaluate the performance of a predictor. This object is part of the <a>WindowSummary</a> object.</p>"
@@ -2246,6 +2270,16 @@
       "max":3000,
       "min":1
     },
+    "OptimizationMetric":{
+      "type":"string",
+      "enum":[
+        "WAPE",
+        "RMSE",
+        "AverageWeightedQuantileLoss",
+        "MASE",
+        "MAPE"
+      ]
+    },
     "ParameterKey":{
       "type":"string",
       "max":256,