Patient Data ETL (Extract, Transform, Load) and Analysis Platform

Challenges

  • Patients’ data is often duplicated in multiple systems, solution needs to find corresponding records and properly attach them to a single master source.
  • Data may have typos, incorrect values in key identifiers – social security number, DOB, HICN, etc.
  • Limited amount of time to download and transform large amount of data (gigabytes) on a daily basis.
  • Mixture of source systems: different versions of SQL Server, FoxPro, custom APIs and CSV uploads in different formats.
  • Data normalization to achieve field-level data consistency. I.e. same hospital or doctor may be spelled differently in different source systems.

Solution

Langate developed a custom solution which extracts, transforms and loads patient data into the central data warehouse. Key deliverables:
  • Scalable data processing workflows which run on multiple servers in parallel.
  • Automatic intelligent data analysis and matching algorithm tolerant of data variance.
  • Configurable semi-automatic mapping engine for data normalization.

Results

  • The resulting solution produces point of reference patient information as a superset of data imported from different systems.
  • All non-reconciled data mistakes and discrepancies are automatically reported to the responsible party.
  • APEX was able to significantly reduce the amount of time spent on manually maintaining patients’ data.

Contact

Us