More control of application instances

[felipmiguel]

Is your feature request related to a problem? Please describe.

Azure Spring Apps application instances cannot be controlled, for instance it is not possible to reboot or kill a specific application instance.
I had an application with some instances failing due to failing dependency. The application requires a connection to a Postgresql server and there were no more available connections when some application instances started. Once fixed the connections availability it was not possible to reboot the failed instances.
The application remains in update or failed stated and didn't allow to do a manual scale-in.
The only way to fix this situation was to completely stop the application, with a service disruption, and then start the application again.
Another impact is that seems that failed instances were charged to the bill, so there costs charged for failed instances that was not possible to stop.

Describe the solution you'd like
Provide a restart action for application instances. For instance in the CLI or Azure Portal, provide an action to restart an application.
Provide a delete action for application instances. That would automatically scale-in the application, reducing the amount of instances.
Describe alternatives you've considered

The only solution found was stopping the application, fix the issue, and start again. It implies a service disruption.

Describe the Customer Impact

As described above, the only way to fix it is by stopping the application with a service disruption.
Failed application instances are billed and there is no way to stop them without service disruption.

Additional context

Add any other context or screenshots about the feature request here.

Can we contact you for additional details? Y/N
Y
If yes, please send us your contact information to [email protected] and include the issue number in the email title.

[taoxu0903]

@felipmiguel just wonder: if the app met DB connection failure due to DB connection unavailability at that moment, supposedly any follow-up DB queries from that app to the DB should be received on DB side and be judged against current available DB connections there. Why not work in that way? the app should have the capability to keep resilient with its backing services, any of which can fall into unavailability situation any time right?

[felipmiguel]

@taoxu0903 the problem is due to the fact that the connections to the database are created during the startup of the application depending on the connection pool configuration. If it is not able to open a given amount of connections it fails and the application do not start. It is also due to the fact the backend database has a limited number of physical connections, in this case postgresql is configured with 5000 max connections. I had an application with multiple instances, the latest instances to start were impacted because there was no more physical connections, so they couldn't start.
I hope it clarifies the scenario, let me know if you need more details.

[allxiao]

We created a work item internally to track this "restart a specific instance" request. It's currently in our backlog.

Currently, the service provides a "Restart" operation for app (in portal or in CLI az spring app restart --help), which allows you to restart a specific deployment in a rolling manner. As long as the health probes and readiness probes are properly defined, this should be a smooth operation without disruption.

For this particular case, a possible way may be treating the "database connection readiness" as a precondition during application start, and fail fast if the connection is not available or broken. When the app instance is failed, the backend will restart this particular instance. So when you fixed the connection limits, it will start normally and go back working.

For "Provide a delete action for application instances. That would automatically scale-in the application, reducing the amount of instances.", it will not be supported. We cannot combine a deletion to a specific instance and a scale-in operation.