Share via

Empty Files being Placed when using OIC(Oracle Integration Cloud) to copy data into Azure Data Lake

Dravya Jain 5 Reputation points
2025-09-05T07:34:28.1466667+00:00

We are currently facing an issue in our UAT environment while copying files from Oracle Integration Cloud (OIC) to Azure Data Lake Gen2 using the OIC connectivity agent installed on an Azure VM inside a secure VNET.

Issue Observed

When the file copy process is triggered, the first file is copied successfully with data into the Data Lake.

However, on concurrent calls, empty files are created in the Data Lake.

No errors are recorded in the OIC logs during the failure.

This behavior is only observed in the UAT environment.

Environment Setup

UAT:

Separate VNET, VM, and Data Lake instance.

  All necessary firewall rules and private endpoint accesses are in place.
  
     OIC Agent is installed on an Azure VM inside this secured network.
     
     **DEV**:
     
        Same architecture and configuration.
        
           File copy works **without any issues** (all files contain data).

Key Points

The issue is reproducible only in UAT (not in DEV).

The first file always contains data, but all subsequent concurrent calls result in empty files.

OIC Agent logs do not show any errors or failures during the process.

All firewall and network access requirements have been validated and are open.

Request

We would like assistance in understanding:

Why the same configuration works in DEV but fails in UAT.

Whether there are any known issues or limitations in handling concurrent file writes from OIC to Azure Data Lake via the agent.

  • Any additional logging or diagnostics recommended to identify the root cause of empty file creation.We are currently facing an issue in our UAT environment while copying files from Oracle Integration Cloud (OIC) to Azure Data Lake Gen2 using the OIC connectivity agent installed on an Azure VM inside a secure VNET. Issue Observed
    • When the file copy process is triggered, the first file is copied successfully with data into the Data Lake.
    • However, on concurrent calls, empty files are created in the Data Lake.
    • No errors are recorded in the OIC logs during the failure.
    • This behavior is only observed in the UAT environment.
    Environment Setup
    • UAT:
      • Separate VNET, VM, and Data Lake instance.
      • All necessary firewall rules and private endpoint accesses are in place.
      • OIC Agent is installed on an Azure VM inside this secured network.
    • DEV:
      • Same architecture and configuration.
      • File copy works without any issues (all files contain data).
    Key Points
    • The issue is reproducible only in UAT (not in DEV).
    • The first file always contains data, but all subsequent concurrent calls result in empty files.
    • OIC Agent logs do not show any errors or failures during the process.
    • All firewall and network access requirements have been validated and are open.
    Request We would like assistance in understanding:
    • Why the same configuration works in DEV but fails in UAT.
    • Whether there are any known issues or limitations in handling concurrent file writes from OIC to Azure Data Lake via the agent.
    • Any additional logging or diagnostics recommended to identify the root cause of empty file creation.
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Kalyani Kondavaradala 3,145 Reputation points Microsoft External Staff Moderator
    2025-09-05T11:18:17.61+00:00

    Hi Dravya Jain,

    Thank you for posting your query on Microsoft Q&A!

    Based on your description i understand that while copying data to Dev, its successfully copied all the files but not copying all files in UAT environment. Please correct me if my understanding is wrong.

    can you please clarify me the below questions so that i can guide you in right path.

    • Does the UAT Data Lake storage account have the same access control settings (RBAC or ACLs) as DEV?
    • What type of files are you copying to ADLS Gen2 (for example: CSV, JSON, Parquet, etc.)?

    In the meantime, here are some common reasons why empty files may appear in ADLS Gen2:

    • The source system returns no data for concurrent calls, but OIC still creates the file placeholder.

    Why It Works in DEV but Fails in UAT

    • Workload Differences: DEV environments typically have lower traffic and fewer concurrent processes. It's likely that in DEV, the concurrent calls are not happening quickly enough to overlap, so the first write completes before the next one starts. In UAT, which is designed to mimic a production workload, multiple processes are hitting the target directory at almost the exact same time, creating a race condition.
    • Networking Latency: Even with "the same" VNET and firewall rules, network latency and throughput can vary between environments. Slower latency in UAT could be causing the first process to take just long enough to acquire a lock on the file path, and before it releases it, the subsequent concurrent processes attempt to write, resulting in a failed write operation that creates an empty file.
    • Azure Data Lake Gen2 Throttling: Azure Data Lake Gen2 has its own service limits and throttling policies. If the UAT environment is under a heavier load than DEV, it could be hitting these limits, causing requests to fail or return without writing data. This might not be an explicit error but a "soft failure" that the OIC agent doesn't log as a critical error.

    Once again please verify your configurations for OIC Agent in Dev and UAT and make sure you have same privileges on both environments.

    Please let me know if you have any further questions on this, happy to assist you.

    Kindly consider upvoting and Accept answer, if the information provided is helpful. This can assist other community members in resolving similar issues.

    Thanks,

    Kalyani

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.