Issues with Azure recognizing which row text corresponds to in table fields

Hello Will B,

Welcome to Microsoft Q&A and Thank you for sharing the details with us.

I understand that you’re experiencing challenges with your custom model not accurately parsing table fields for example, assigning the same value to multiple rows. This is a common issue when working with scanned documents, complex layouts, or OCR noise, and it can definitely be frustrating especially when retraining the model doesn’t yield the expected improvements.

Ensure that the quality and consistency of your training data are as high as possible. Any unexpected characters, formatting issues, or low-quality scans can confuse the model and lead to incorrect associations.

It’s important that your dataset includes a diverse range of table structures including merged cells, varying row heights, and edge cases with overlapping characters so the model can learn to handle them effectively.

Since you mentioned that overlapping characters might be affecting row detection, it’s worth focusing on preprocessing your input documents before sending them to the model.

Enhancing image quality, standardizing contrast, aligning tables, or slightly increasing row spacing can improve OCR accuracy. Additionally, if character tails (like “y” or “g”) are extending into adjacent rows, you can try noise removal or image segmentation techniques to reduce their impact. This step often makes a significant difference in how accurately the model associates text with the correct rows.

When labeling your data, make sure the bounding boxes for each field are tightly drawn around the relevant text and do not overlap with adjacent rows. Overlapping annotations often lead to repeated or misplaced values.

You can also add contextual anchors like row indices or column headers during labeling, which helps the model better understand table structure and row relationships.

Since you’ve already retrained the model multiple times, it may help to experiment with different training configurations or adjust model parameters. In some cases, training a separate custom model that focuses only on table extraction (rather than mixing tables with other layouts) can improve accuracy. Specialized models often perform better because they learn structure-specific patterns more effectively.

If accuracy is still not improving, consider adopting a continuous learning approach. This means validating the model’s predictions regularly, identifying where it fails, and retraining it with those new examples. Over time, the model will learn from its mistakes and become more robust against edge cases and layout variations.

Adding a post-processing layer after extraction can further improve reliability. You can implement logic to validate and correct repeated or inconsistent values based on row context, merge or split cells programmatically, or cross-check extracted fields with known formatting patterns. These checks act as a safety net to catch errors that the model might miss.

By improving data quality, refining preprocessing, experimenting with model tuning, and introducing continuous learning and post-processing logic, you can greatly enhance your custom model’s ability to correctly associate text with the right rows. Please try these steps and let us know how it goes.Please refer this Document Intelligence custom models.

I Hope this helps. Do let me know if you have any further queries.

Thank you!

Hello Will B,

Did you get any chance to review the above response. Do let me know if you have any further queries.

Thank you!

 Hello Will B,

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.

Thank you!

Hello SRILAKSHMI C,

Thank you for your help! I appreciated the detailed response. And sorry for the delay!

We were already trying some of these techniques, and are incorporating other suggestions too now. That said we haven't noticed much of a difference. I think we will try different post-processing strategies next.

Ultimately, it seems like the best thing to do would be to give more room in the rows of the table so no text crosses lines. Unfortunately we do not have control over the designs used.

Hi Will B,

Thank you for the update and for sharing the context. I completely understand the challenges you’re facing, especially since the table designs are out of your control. When text from one row overlaps into the next, even the most well-trained model can struggle to correctly associate values.

Since modifying the original table layout isn’t possible, the most effective strategies at this stage are likely focused on preprocessing and post-processing:

1. Image Preprocessing:

• Apply techniques to clean the scanned documents, such as increasing contrast, binarization, or using OCR-specific noise reduction.

Deskewing or slightly adjusting the alignment of the tables can help separate overlapping characters.

Sometimes splitting pages into smaller table segments or cropping rows individually before sending them to the model improves row recognition.

2. Post-Processing:

Implement row validation logic to detect repeated values or improbable duplicates. For example, if “Will” appears in two consecutive rows in the first name field, the second occurrence can be flagged for review or corrected based on context.

Use heuristics or patterns (like expected row lengths, data types, or unique identifiers) to resolve ambiguities in the extracted data.

Consider applying row segmentation after OCR: detect table lines and reassign text blocks to the correct row programmatically before further processing.

3. Continuous Learning:

Continue to retrain the model with examples of these edge cases, but supplement this with automated corrections via post-processing. Over time, this combined approach can significantly improve reliability.

4. Explore Specialized Table Models:

If feasible, creating a dedicated model trained solely on table layouts similar to your edge cases can improve row association accuracy. Models trained on mixed layouts sometimes underperform on tight or overlapping tables.

Ultimately, in scenarios where text overlaps between rows and design control is limited, a combination of robust preprocessing, targeted post-processing, and specialized table-focused training usually yields the best results.

Thank you!