How to run SQL commands to create or update a database in an automated manner on the AWS RDS

We have set multi-tenant applications, where we have AWS RDS databases set up for each client, so we have 1 DB for 1 client setup.

When we have a client to be set up we need to create a starter DB and then run a few scripts to set it up. We are actually migrating the app from azure to AWS and in Azure, this is achieved by running a few PowerShell scripts manually when we have a request to set up a new client.

Now we are migrating this to AWS and we are thinking to achieve in an automated way, and we are looking for the best suitable managed service to leverage (we are against using EC2 or any unmanaged service to achieve this) and achieve.

Currently, we are thinking to have the SQL commands in a bash script and run it via Lambda, but we are not sure Lambda supports running Bash scripts and we want to pass some parameters as well, like client name, etc. If anyone has achieved this and/or has solved a similar problem would request help and is very much appreciated.

Thanks