Recommended model configurations

A recommended model configuration defines the default values that CosmicAC uses to serve a model in a Managed Inference Job. It stores the model's runtime image and serving parameters, so you enter fewer values for each job.

How a job uses a recommended model configuration

When you select a model while you create a Managed Inference Job, CosmicAC loads that model's recommended configuration and uses its values to prefill Serving configuration.

You can change the prefilled serving parameters before you create the job. In the web interface, you can't change the runtime image. CosmicAC uses the runtime image from the model's recommended configuration.

A recommended model configuration also sets which serving parameters raise a warning when you create a job. For each parameter, the configuration sets whether the warning appears when the job's value is lower than, higher than, or different from the configuration's value.

A vLLM Managed Inference Job can serve any model that vLLM supports, but the job creation form lists only the models that have a recommended model configuration.

Ways to create a recommended model configuration

You can create a recommended model configuration in two ways.

• Models > Recommended configurations > Add new: create the configuration and enter each value.
• Models > Register model: register a vLLM model. If the model has a published vLLM recipe, CosmicAC can prefill the values from it.

Either way, you can later update or archive the configuration.

Where you manage recommended model configurations

You manage recommended model configurations on the Models page in the web interface. The following guides cover each task.

For the recommended values for a set of models, see Recommended configuration values.