How to add properties when running batch endpoints using REST API?.

Question

How to add properties when running batch endpoints using REST API?.

Guilherme Matheus 120 Microsoft Employee

We currently have a pipeline in ADF that we are using to train a given model using Pipelines components, and if we check this doc PipelineJob we can see the properties available, that way I am being able to use displayName, tags, etc normally for example.

Now, we are trying to do the same in ADF pipeline for Batch Endpoints by following this guide Run batch endpoints from Azure Data Factory which leverages this api https://<endpoint_name>.<region>.inference.ml.azure.com/jobs but I cannot see which properties are available for this endpoint in this case, so my pipeline displayName are coming with default/random name which we do not want:

User's image

Is there any documentation providing this info? How can we achieve this?

Danny Nguyen (WICLOUD CORPORATION) 3,515 Reputation points Microsoft External Staff

2025-10-08T05:30:10.9166667+00:00
Hi,

You can pass a custom properties dictionary when invoking a Batch Endpoint job (these show up under “Properties” in the job pane), but as of now you cannot set or override the job’s display name, tags, or explicit job name in the single invoke call. The service auto‑generates the display name.

What works in the invoke payload:

inputs / (optionally) outputs

resources & settings (mini_batch_size, concurrency, retries, etc.)

environment_variables

properties: { "anyKey": "anyValue", ... } (your custom metadata / correlation IDs)

Hope this helps.
Swapnesh Panchal 740 Reputation points Microsoft External Staff Moderator

2025-10-09T19:24:12.6633333+00:00

Hi Guilherme Matheus,
Hope you’ve had a chance to check the info provided by Danny Nguyen. If you still have questions, just drop them in the comments
Guilherme Matheus 120 Reputation points Microsoft Employee

2025-10-10T12:39:25.1266667+00:00

I saw his answer, but I am trying a few things still, because for some reason I have tags now, and I think is possible to do something. I am not really confident about this yet.
Swapnesh Panchal 740 Reputation points Microsoft External Staff Moderator

2025-10-10T13:51:43.79+00:00

Hi Guilherme Matheus,
Thanks for the update. Seeing the tags is a good sign. Let us know if you need further communication or assistance, or if you have additional questions—I'm happy to discuss.
Guilherme Matheus 120 Reputation points Microsoft Employee

2025-10-10T23:07:21.26+00:00
@Swapnesh Panchal @Danny Nguyen (WICLOUD CORPORATION) Seems like indeed it doesn't work. But this is terrible to be honest, because lots of pipelines/jobs/experiments are generated with a random name, and it's not near to be organized, different than jobs that I have shared earlier. But I have few questions please:

Is there a work around?

Who may I contact internally since I am a MSFT employee, to request this feature?

Similar to job documentation, is there any documentation for inference API, where I can check the payload? This is also very important
Guilherme Matheus 120 Reputation points Microsoft Employee

2025-10-13T02:21:39.69+00:00

Another question, is there a way to select an existing environment when doing inference through ADF using that API? Otherwise, a new environment will be created every single time?
Swapnesh Panchal 740 Reputation points Microsoft External Staff Moderator

2025-10-15T18:59:46.4066667+00:00

Hi Guilherme Matheus,

• Batch Endpoint invoke (…inference.ml.azure.com/jobs) auto-generates the job name/displayName.
• You cannot set displayName, name, or tags in the invoke call (by design).
• You can send: inputs/outputs, runtime settings (mini_batch_size, concurrency, retries), environment_variables, and a custom properties dict for metadata/correlation IDs.
• The environment used at runtime comes from the deployment. If you see “Anonymous…”, the deployment was created with an inline env. To control it, update/recreate the deployment to use a named Environment.
• If you need to control display name or tags, use the Jobs API or submit a Pipeline Job (those payloads support displayName and tags).
We can also do this
– Put adfRunId, pipeline name, etc., under properties for tracking.
– Standardize organization with a consistent properties schema (team, env, app).
Swapnesh Panchal 740 Reputation points Microsoft External Staff Moderator

2025-10-16T22:09:22.6533333+00:00

Hi Guilherme Matheus,
Hope you’ve had a chance to look over the comments we shared earlier and that it was helpful! If you still have any questions, please feel free to share them in the comments — we’d be happy to help.

1 answer

Your answer

Danny Nguyen (WICLOUD CORPORATION) 3,515 Reputation points Microsoft External Staff

2025-10-08T05:30:10.9166667+00:00

Hi,

You can pass a custom properties dictionary when invoking a Batch Endpoint job (these show up under “Properties” in the job pane), but as of now you cannot set or override the job’s display name, tags, or explicit job name in the single invoke call. The service auto‑generates the display name.

What works in the invoke payload:

inputs / (optionally) outputs

resources & settings (mini_batch_size, concurrency, retries, etc.)

environment_variables

properties: { "anyKey": "anyValue", ... } (your custom metadata / correlation IDs)

Hope this helps.
Swapnesh Panchal 740 Reputation points Microsoft External Staff Moderator

2025-10-09T19:24:12.6633333+00:00

Hi Guilherme Matheus,
Hope you’ve had a chance to check the info provided by Danny Nguyen. If you still have questions, just drop them in the comments
Guilherme Matheus 120 Reputation points Microsoft Employee

2025-10-10T12:39:25.1266667+00:00

I saw his answer, but I am trying a few things still, because for some reason I have tags now, and I think is possible to do something. I am not really confident about this yet.
Swapnesh Panchal 740 Reputation points Microsoft External Staff Moderator

2025-10-10T13:51:43.79+00:00

Hi Guilherme Matheus,
Thanks for the update. Seeing the tags is a good sign. Let us know if you need further communication or assistance, or if you have additional questions—I'm happy to discuss.
Guilherme Matheus 120 Reputation points Microsoft Employee

2025-10-10T23:07:21.26+00:00

@Swapnesh Panchal @Danny Nguyen (WICLOUD CORPORATION) Seems like indeed it doesn't work. But this is terrible to be honest, because lots of pipelines/jobs/experiments are generated with a random name, and it's not near to be organized, different than jobs that I have shared earlier. But I have few questions please:

Is there a work around?

Who may I contact internally since I am a MSFT employee, to request this feature?

Similar to job documentation, is there any documentation for inference API, where I can check the payload? This is also very important
Guilherme Matheus 120 Reputation points Microsoft Employee

2025-10-13T02:21:39.69+00:00

Another question, is there a way to select an existing environment when doing inference through ADF using that API? Otherwise, a new environment will be created every single time?
Swapnesh Panchal 740 Reputation points Microsoft External Staff Moderator

2025-10-15T18:59:46.4066667+00:00

Hi Guilherme Matheus,

• Batch Endpoint invoke (…inference.ml.azure.com/jobs) auto-generates the job name/displayName.
• You cannot set displayName, name, or tags in the invoke call (by design).
• You can send: inputs/outputs, runtime settings (mini_batch_size, concurrency, retries), environment_variables, and a custom properties dict for metadata/correlation IDs.
• The environment used at runtime comes from the deployment. If you see “Anonymous…”, the deployment was created with an inline env. To control it, update/recreate the deployment to use a named Environment.
• If you need to control display name or tags, use the Jobs API or submit a Pipeline Job (those payloads support displayName and tags).
We can also do this
– Put adfRunId, pipeline name, etc., under properties for tracking.
– Standardize organization with a consistent properties schema (team, env, app).
Swapnesh Panchal 740 Reputation points Microsoft External Staff Moderator

2025-10-16T22:09:22.6533333+00:00

Hi Guilherme Matheus,
Hope you’ve had a chance to look over the comments we shared earlier and that it was helpful! If you still have any questions, please feel free to share them in the comments — we’d be happy to help.

Answer 1

Hi Guilherme,

The PipelineJob document is used for creating jobs in Azure ML Workspace. Among various jobs in ML Workspace one of the job is called pipeline jobs.

As you are using Run batch endpoints from Azure Data Factory but the inference endpoint https://<endpoint_name>.<region>.inference.ml.azure.com/jobs is invoking job from azure ml workspace.

An ML pipeline job is different from Azure Data Factory Pipeline.

For Azure Data Factory Pipeline below is the relevant documentation:
https://free.blessedness.top/en-us/rest/api/datafactory/pipelines/create-run?view=rest-datafactory-2018-06-01&tabs=HTTP

You can use batch endpoints in Azure Data Factory but to deploy batch endpoints you have to use Azure ML Workspace.

To create jobs and input data for batch endpoints in Azure ML Workspace, here is relevant documentation:
https://free.blessedness.top/en-us/azure/machine-learning/how-to-access-data-batch-endpoints-jobs?view=azureml-api-2&tabs=rest

You can add properties when creating or updating a job, here is relevant documentation:
https://free.blessedness.top/en-us/rest/api/azureml/jobs/create-or-update?view=rest-azureml-2025-09-01&tabs=HTTP#createorupdate-pipeline-job.

To update the name here is the YAML schema documentation:
https://free.blessedness.top/en-us/azure/machine-learning/reference-yaml-job-pipeline?view=azureml-api-2#yaml-syntax
In this schema if you don't provide name then a random name will be generated as you shared in the screenshot.

To manage inputs and outputs for components and pipelines, here is relevant documentation:
https://free.blessedness.top/en-us/azure/machine-learning/how-to-manage-inputs-outputs-pipeline?view=azureml-api-2&tabs=cli

Feel free to accept this as an answer.
Thankyou for reaching out to The Microsoft QNA Portal.

Guilherme Matheus 120 Reputation points Microsoft Employee

2025-10-28T12:21:11.15+00:00

Thanks for providing all the links, it's really helpful. I am able to submit AML training job using WEB activity with this URL https://management.azure.com/subscriptions/d0ec0b8c-1066-40ab-8ae7-2719855f16f8/resourcegroups/gravity-gmathe-us-01-regional/providers/Microsoft.MachineLearningServices/workspaces/gmatheus01rmlw/jobs/gmatheus01rmlwcompelasticvm_ef545a5d-a915-4d9e-88ba-635fe3668ac6?api-version=2025-06-01 and it's working fine, I even add properties normally.

As you also noticed, I am also submitting a inference job in AML from ADF which is working perfect fine as well. But, my only question is HOW to add these properties (displayName, experimentName, tags) for this API also. Because from the second link, which is explaning how to submit inference using ADF there is another link showing the properties: https://free.blessedness.top/en-us/azure/machine-learning/how-to-access-data-batch-endpoints-jobs?view=azureml-api-2&tabs=rest#configure-the-experiment-name, but this is not working for me, as it is described in the documentation:

"properties": { "experimentName": "my-batch-job-experiment" }

So if we are in fact submitting a job using API, how may I use those properties?

Share via

How to add properties when running batch endpoints using REST API?.

1 answer

Your answer