Regression migrating from Standard to Enterprise Tier: --runtime-version not supported with app deployment create

[ezYakaEagle442]

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:

See my Workflow deployment at https://github.com/ezYakaEagle442/azure-spring-apps-petclinic-mic-srv/blob/azure/.github/workflows/build-deploy-apps-staging-CLI.yml#L467

  DEPLOYMENT_RUNTIME_VERSION: Java_11
  DEPLOYMENT_CPU: 500m
  DEPLOYMENT_MEMORY: 1Gi
  DEPLOYMENT_INSTANCE_COUNT: 1
  
          az spring app deployment create \
            --name $DEPLOYMENT_NAME \
            --app ${{ env.VETS_SERVICE }} \
            --service ${{ env.AZURE_SPRING_APPS_SERVICE }} -g ${{ env.RG_APP }} \
            --instance-count ${{ env.DEPLOYMENT_INSTANCE_COUNT }} \
            --cpu ${{ env.DEPLOYMENT_CPU }} \
            --memory ${{ env.DEPLOYMENT_MEMORY }} \
            --version ${{ env.DEPLOYMENT_VERSION }} \
            --runtime-version ${{ env.DEPLOYMENT_RUNTIME_VERSION }} \
            --skip-clone-settings

--runtime-version doesn't support for Enterprise tier Spring instance.

Expected behavior
The Workflows & az spring app deployment create should run as is with Standard tier

Screenshots
N/A

Additional context
see also MicrosoftDocs/azure-docs#102826 (comment)

[ezYakaEagle442]

in the docs at https://learn.microsoft.com/en-us/azure/spring-apps/quickstart-deploy-apps-enterprise#deploy-polyglot-applications-with-tanzu-build-service I notice thye usage of BP_JVM_VERSION env. var :

az spring app deploy \
    --resource-group <resource-group-name> \
    --name payment-service \
    --service <Azure-Spring-Apps-service-instance-name> \
    --config-file-pattern payment/default \
    --source-path apps/acme-payment \
    --build-env BP_JVM_VERSION=17

This is mentioned at https://learn.microsoft.com/en-us/azure/spring-apps/how-to-enterprise-build-service?tabs=azure-portal :

_If you're using the tanzu-buildpacks/java-azure buildpack, we recommend that you set the BP_JVM_VERSION environment variable in the build-env argument.

When you use a custom builder in an app deployment, the builder can't make edits and deletions. If you want to change the configuration, create a new builder. Use the new builder to deploy the app.

After you deploy the app with the new builder, the deployment is linked to the new builder. You can then migrate the deployments under the previous builder to the new builder, and make edits and deletions._

ALL Build Pack env vars usage/purpose should be documented in a dedicated section, not in a small section.
Also it should give pointers to the official docs from :

• https://docs.vmware.com/en/VMware-Tanzu-Buildpacks/services/tanzu-buildpacks/GUID-java-native-image-java-native-image-buildpack.html
• https://buildpacks.io/docs/app-developer-guide/ ==> I have NOT found any reference to BP_JVM_VERSION there

[ezYakaEagle442]

I have updated the az spring app deploy command :

  BP_JVM_VERSION: Java_11 #
  BUILD_ENV: BP_JVM_VERSION=Java_11 
  BUILD_CPU: 1 # CPU resource quantity. Should be 500m or number of CPU cores. Default: 1.
  BUILD_MEMORY: 2Gi # Memory resource quantity. Should be 512Mi or #Gi, e.g., 1Gi, 3Gi. Default: 2Gi.
  BUILDER: default # java-builder
  CFG_FILE_PATTERNS: application 
  
        az spring app deploy \
        --name ${{ env.VETS_SERVICE }} \
        --service ${{ env.AZURE_SPRING_APPS_SERVICE }} -g ${{ env.RG_APP }} \
        --artifact-path ${{ env.VETS_SERVICE_PACKAGE_PATH }} \
        --deployment $DEPLOYMENT_NAME \
        --disable-probe true \
        --env SPRING_CLOUD_AZURE_KEY_VAULT_ENDPOINT=${{ env.SPRING_CLOUD_AZURE_KEY_VAULT_ENDPOINT }} VETS_SVC_APP_IDENTITY_CLIENT_ID=${{ env.VETS_SVC_APP_IDENTITY_CLIENT_ID }} SPRING_CLOUD_AZURE_TENANT_ID=${{ env.SPRING_CLOUD_AZURE_TENANT_ID }} \
        --jvm-options="${{ env.DEPLOYMENT_JVM_OPTIONS }}" \
        --version ${{ env.DEPLOYMENT_VERSION }} \
        --build-env ${{ env.BUILD_ENV }}  \
        --build-cpu ${{ env.BUILD_CPU }} \
        --build-memory ${{ env.BUILD_MEMORY }} \
        --builder ${{ env.BUILDER }} \
        --config-file-patterns ${{ env.CFG_FILE_PATTERNS }}

But the deployment sill fails with :

12/18/2022, 1:18:23.989 PM
vets-service
all memory regions require 1774834K which is greater than 1G available for allocation: -Xmx1G, 0 headroom, -XX:MaxDirectMemorySize=10M, -XX:MaxMetaspaceSize=214258K, -XX:ReservedCodeCacheSize=240M, -Xss1M * 250 threads
12/18/2022, 1:18:23.989 PM
vets-service
unable to calculate memory configuration
12/18/2022, 1:18:23.778 PM
vets-service
Adding $JAVA_OPTS to $JAVA_TOOL_OPTIONS
12/18/2022, 1:18:23.770 PM
vets-service
Setting Active Processor Count to 16

[allxiao]

--runtime-version is a parameter used in Standard / Basic tier, to specify the Java version when you upload your JAR artifacts for deployment.

As you had already found, in Enterprise, the application source code are built to an OCI image, where the Java version is determined by the BP_JVM_VERSION passed to the build process. We do need some more documentation to clarify the build environment variable usages.

For the final error

all memory regions require 1774834K which is greater than 1G available for allocation: -Xmx1G, 0 headroom, -XX:MaxDirectMemorySize=10M, -XX:MaxMetaspaceSize=214258K, -XX:ReservedCodeCacheSize=240M, -Xss1M * 250 threads

It seems you specified -Xmx1G in JAVA_TOOL_OPTIONS environment variable or JVM options, while configured the app to have 1G memory. As the memory calculator said, the maximum memory that can be required potentially is greater than 1G, which is likely to leads to OOM Killed later at runtime. The memory calculator runs before the application is started to 1) setup the optimized memory options for JVM and 2) fail fast if it detects any incorrectly configured memory settings.

You can try to remove the memory settings from JAVA_TOOL_OPTIONS or JVM options and let the memory calculator to set up the optimized default based on the app memory size.