Human in the Loop Document Review UI (Architecture on AWS and MongoDB)

Key Takeaways

• Human in the loop review is the fastest way to reach production reliability.
• Keep the UI focused on exceptions, not full document editing.
• Store provenance and audit logs so you can defend every value.
• Use validation gates to control review workload.

Fully automated document extraction is rare in real operations. The difference between a pilot and a production system is usually a clean human in the loop layer that handles exceptions quickly.

This post describes a pragmatic review UI architecture using AWS for storage and compute, and MongoDB for workflow state.

Why human in the loop is a feature, not a failure

Human review is what lets you:

• Ship sooner, before you have perfect training data
• Control risk on sensitive fields
• Improve continuously through feedback loops

What the review UI should actually do

A useful review UI is focused:

• Show the original page and the extracted fields side by side
• Highlight citations and bounding boxes
• Let reviewers correct fields with minimal clicks
• Capture reason codes for failures
• Support queues and assignments

Reference architecture

Storage

• Raw docs in S3
• Normalized page images in S3
• Evidence and redlines in S3

S3 reference: https://docs.aws.amazon.com/AmazonS3/latest/userguide/Welcome.html

API and workflow

• Step Functions for orchestration
• API service for task assignment and updates

References:

• Step Functions: https://docs.aws.amazon.com/step-functions/latest/dg/welcome.html
• AWS Lambda: https://docs.aws.amazon.com/lambda/latest/dg/welcome.html

MongoDB collections

• review_tasks: status, assignedTo, dueAt
• extracted_fields: field values, confidence, citations
• review_actions: diffs, timestamps, reviewer ID
• audit_log: append only

MongoDB change streams are useful to push UI updates:

• https://www.mongodb.com/docs/manual/changeStreams/

Quality gates and confidence thresholds

Two simple gates reduce review volume:

• Only send to review when a required field is missing or below confidence threshold
• Auto approve when validation rules pass

Security and access control

Review systems often contain PII.

Baseline controls:

• Strong authentication and RBAC
• Per tenant authorization checks
• Encrypted storage and short lived signed URLs

References:

• OWASP ASVS: https://owasp.org/www-project-application-security-verification-standard/
• AWS KMS: https://docs.aws.amazon.com/kms/latest/developerguide/overview.html

zion OCR demo: https://ocr.lidvizion.ai/

FAQs

Q: How many reviewers do we need? Start by measuring the exception rate. A good system reduces review volume over time through validation and model improvements.

Q: Can we use reviewers to improve models? Yes. Store corrected values with citations and use them as labeled data for retraining or rule refinement.

Q: How do we prevent reviewers from seeing data they should not? Enforce tenant checks in the API, and limit S3 signed URL scope and lifetime.

Internal reference:

• Lid Vi

Q: What should we link to internally? A: Link to relevant solution pages like Computer Vision or Document Intelligence, and only link to published blog URLs on the main domain. Avoid staging links.