PRS

1.1.1 HN202605 - Support true merge/unmerge of duplicate providers
This change request is part of the employed providers enhancement.
If duplicate providers are inadvertently created, they can be merged. This enhancement provides a way to merge one provider with another. In release 2.05, duplicate providers were linked, but in release 3, the design is to provide a “deep” merge of one provider with another.
1.1.1.1 Requirement(s)
This enhancement is related to the following Release 3 enhancements:
• HN-202670 Employed Provider
• HN-202670 Enhanced Searching and Matching
For Employed providers, there will be many occasions where the same individual (person) holds several jobs in more than one agency. Furthermore, it is likely that many organizations will be Primary Sources for the same Provider Type.
It is likely that uploads or XML messages from multiple sources will contain data about the same person, in a single role or in different roles.

PRS’s ability to avoid creating duplicates during the creation of providers was enhanced in Release 4. Despite enhanced searching and matching functionality, it is expected that duplicates will be created and that the number of duplicates may increase. PRS’s ability to resolve duplicates was therefore enhanced.

Duplicate resolution includes the ability to reference (or associate) each unique identifier with any other unique identifier (CPN). This is to accommodate duplicate record errors since unique identifiers cannot be deleted. Therefore a user should be able to look at any unique identifier record and determine all instances where this identifier is referenced to or associated with any other unique identifier. Also if two records have been related in error there needs to be the ability to un-relate or disassociate the two records.

The unique identifier should be sharable, and available for access and use by all users.
Implementation of these requirements must also support the following merge rules:
• Duplicates are never deleted but are ceased and the ceased record points to the non-ceased record (forward referencing)
• Resolution of Duplicate Providers must be done manually; that is, a person must make the decision as to which record to retire/cease—this cannot be done automatically by PRS
• Each merge decision must be reversible
1.1.1.2 Analysis
In Release 3, PRS supported only linking providers with the same CPN. The Release 3 link process was completed through the Resolve Duplicate Search Results Screen function. Selecting ‘duplicate’ Provider records and pressing the “Merge” button results in PRS changing the CPN for the selected duplicate Providers to that of the ‘target’ Provider. The CPNs of the duplicate Providers are in effect ‘ceased’. The ceased CPNs are no longer accessible—that is, no history is kept on ceased CPNs that is viewable by Registry Users. No history on CPNs means that it is difficult to track if two Providers have been linked in error. Furthermore, there is no ‘unlink’ functionality in PRS if two Providers have been linked in error.
The crux of the problem in linking Providers is that in the real world, at any one time, a Provider can be linked to only one party, but over time, the provider might be related to several. Although the PRS database tracks history, in Release 3 it only allowed for a provider to be linked to one party, so it was unable to maintain details of “CPN History”.

Following are scenarios that illustrate situations that may occur within PRS with the introduction of Employed Providers functionality (Release 4). Merge and Unmerge functionality are addressed within separate subsections.
1.1.1.2.1 Merge
The following scenario illustrates the Release 3PRS duplicates resolution functionality:

Scenario 1: Two sources create two different Providers who are the same Party.



Using the Resolve Duplicate Search Results Screen, the Registry User that is resolving the duplicates needs to cease (Party) CPN 0335 and link to (Party) CPN 0224.



The desired result is that there are now two Providers, Jane MD and Jane Pharmacist under a single CPN 0224.BC.PRS. Although this link functionality is supported in PRS Release 3, what is not supported is forward referencing. That is, if a Registry User subsequently searches for the second Provider (CPN 0335.BC.PRS), Release 3PRS will not point to the first Provider (CPN 0224.BC.PRS).

The following diagram illustrates the result of Scenario 1 with ‘forward referencing’: pointing the ceased Party to the target Party.





Following are two additional situations that may result in Duplicate Providers but unlike Scenario 1, cannot be resolved in Release 3 PRS. Both situations are analysed through scenarios.

• A single source records the same Provider twice on PRS with minor changes to the data elements (e.g. name).
• Two sources create two versions of the same Provider, each with their own data.

Scenario 2: A source records same Provider twice on PRS with minor changes to the data elements (e.g. name).


The Registry User that is resolving the duplicates needs to cease the second Provider entirely, as it was added in error.


The desired result is that the second Provider is ceased. However, if a Registry User subsequently searches for the second Provider (CPN 0335.BC.PRS), PRS will point to the first Provider (CPN 0224.BC.PRS). Pointing to a Provider after ceasing another when they are logically associated with each other (also known as forward referencing) is not supported in Release 3 PRS.

Scenario 3: Two sources create two versions of the same Provider, each with their own data.



Using the Resolve Duplicate Search Results Screen, the Registry User that is resolving the duplicates needs to retain HA2’s data objects (Name, Status, Provider ID, and Work Location) but point to the original Party/Provider. Note that HA2 does not wish to change HA1’s data, but to add their own set of the Provider’s data.



The desired result is that HA2’s data objects are merged with HA1’s data objects under the same Party/Provider. Both ceased Party and Provider point to the target Party and Provider.
1.1.1.2.2 Unmerge
The Release 3 link functionality has been enhanced and expanded in Release 4 to include merge functionality to support the business scenarios above. As stated earlier, there is no ‘unlink’ functionality in Release 3 PRS if two Providers have been linked in error. The scenario below illustrates a business situation that could occur for which unmerge functionality is required.

Scenario 4: Provider data submitted by two different sources was merged in error.

This scenario builds on Scenario 3. An error was made in merging HA2’s data with HA1’s data: Party 0224.BC.PRS and 0335.BC.PRS are, in fact, two different people.


The registry user needs to be able to restore PRS’ data to the state before the merge.



The desired result is that HA2’s Party, Provider and data objects are unmerged from HA1’s. There are now two separate Parties: Jane Brown, Lab Tech submitted by HA1 and Jayne Brown, Lab Tech submitted by HA2 are two different people.
1.1.1.3 Solution
To meet the above requirements and to support the above business scenarios we propose the following enhancements to PRS:
• Modify the database and application design, to support multiple (historical) linkages between a Provider and a CPN
• Modify Resolve Duplicates functionality to implement the 3 merge scenarios outlined above.
• Modify Provider Query functionality to “follow” forward references.
• Modify Provider Maintenance functionality to allow maintenance operations to allow merge and undo-merge operations.
Each enhancement is described in more detail below.
1.1.1.3.1 Support multiple (historical) linkages between a Provider and a CPN
Modify the database design to support multiple linkages between a Provider and a CPN as follows.
In the proposed model, CPNs and IPCs are both regarded as Registry Identifiers. They are subject to permission control, but they cannot be placed in Data Sets.
The database enhancement will be reflected in Provider Query and Provider Maintenance XML schemas, and in the Provider Details/Maintenance Web screen. The following screen shows current CPN and IPC, and historical CPN and IPC, as would be apparent after a scenario 3 merge.

Figure 46 Provider Data and Registry User Identifiers after a Merge
This Provider contains some data that was merged-in from CPN 335. Forward referencing works because a query on CPN 335 will match this provider. Any search on data merged in from the original CPN 335 will also match this provider.
CPN and IPC will be protected by data permission control, and historical display controls (Show Audit, Show History) just like any other object. However, they may only exist in a single “SYSTEM” data set. (The name of this Data Set is never shown in the outline control above).
1.1.1.3.2 Modify Resolve Duplicates Functionality
The Release 3 Resolve Duplicates screen functionality and XML schema will support the functionality required:

Figure 47 Resolve Duplicates Screen
Modification was required on the server side (MaintainPotentialDuplicatesBean) to treat each identified pair as follows. Suppose in the following example that Provider A is to be kept, and Provider B is to be merged/linked.
If Providers A and B have different role types (Scenario 1 merge), perform a LINK as follows:
Cease B’s current CPN
Create a new CPN for B, equal to A’s current CPN
If Candidate and Trigger have the same role type, and were created by the same source (Scenario 2 merge), cease Provider B as follows:
Cease B’s current CPN and IPC
Create new, CEASED CPN and IPC for A, equal to B’s old CPN and IPC
If Candidate and Trigger have the same role type, but were created by different sources (Scenario 3 merge), perform a true merge as follows:
Cease B’s current CPN and IPC
Create new, CEASED CPN and IPC for A, equal to B’s old CPN and IPC
Attempt to copy each of B’s data objects to A. If any business key collisions occur, reject the transaction, listing all collisions in the error response (the Registry Administrator will need to resolve these collisions). This could be achieved by simply creating a MaintainProvider transaction out of B’s data.
1.1.1.3.3 Merging a Merged Provider
There might be a case where a merged provider must be merged again with another provider. Forward referencing will still work in this case. After a merge of two providers, the merged provider (say Doc 2) has effectively disappeared from view because it no longer has a current registry ID. The surviving provider (say Doc 1) has a copy of the merged provider’s CPN as a ceased entry. Now if Doc 1 is to be merged with Doc 3, and Doc 3 is the survivor, then Doc 1 effectively disappears because it will not have a currently active registry ID after the merge. Doc 3 will have a currently active registry ID, and a copy of Doc 1’s registry ID that has been ceased.
1.1.1.3.4 Modifications to ProviderQuery functionality
Provider Queries must be modified in two ways:
• Providers without an active CPN must not normally be returned (regardless of whatever other data are active), except.
• CPN or IPC searches must match INACTIVE CPNs or IPCs. This will allow Provider A (in each merge scenario above) to be returned when Provider B’s CPN or IPC was used as a search criterion.
The first point is important because it allows the Registry Administrator to find providers with no active CPN. In order to assist the Registry Administrator, this special search will be available from the provider menu, beneath “Dupl.” (for “Duplicates”).

Figure 48 Screen Mock-up of Unmerge button
When the Registry Administrator click on the unmerge button, a search screen appears that has a single search query parameter field for the CPN. The unmerge button is controlled by functional permissions and will be visible only to users with that functional permission.
A separate search screen solely for Registry Administrators who need to unmerge providers is an aid to establishing a mental mode of operation that will assist in accurate and careful consideration of unmerge operations.

Figure 49 Special Provider Search screen mock-up
The result of the search will be the provider search results screen. The Registry Administrator should see the providers to be unmerged in the results if the CPN was specified correctly as a search parameter.
The Special Provider Search actually calls the normal provider query method, but sets a parameter that tells the method to disable all other parameters in the search, make the confidence level 100%, and INCLUDE INACTIVE PROVIDERS (ie those without active registry identifiers).
To unmerge, the Registry Administrator identifies the two providers to unmerge, one of which is to be reactivated. The Registry Administrator unceases (recreates) the registry ID from the merged provider. This effectively makes the merged provider visible again. The registry administrator must use the provider details report to compare the two providers. If changes have occurred while the two providers were merged, those changes must be applied to the unmerged provider again. Similarly, any information that was copied to the retained provider from the merged provider must be ceased.
The second point is important because queries can still be conducted successfully by users who have retained old identifiers. Since ceased CPNs and IPCs are searched, and the CPN and IPC of a merged provider became ceased items in the retained provider, the retained provider will be found when an old identifier is used.
The rule that only providers with active CPNs / IPCs are returned by the provider search is enhanced slightly to support unmerged providers. If the same CPN or IPC is found to be ceased in one provider, and active in another, then ONLY the active provider is shown. This prevents two providers being returned after an unmerge, as would happen if the search implemented the search rule without modification.
In summary, the rules for searching are as follows:
1. Exclude from the result set any provider that does not have an active CPN.
If the same CPN/IPC is ceased in one provider and active in another of the same role type, return the active one. Thus ceased CPNs and IPCs do not match if there is an active one for a provider of the same role type.
2. Create a special provider search that allows registry administrators (or those with the correct functional permissions) to search by CPN only for providers with an active or inactive matching CPN.
3. If the CPN is passed as a parameter to the provider search, then all other search parameters are disabled, and the confidence level is always 100%.
1.1.1.3.5 Modifications to Provider Maintenance to support Merge and Unmerge
Provider Maintenance functions were modified in Release 4 to support Merge and Unmerge as follows:
• A Registry Administrator must be able to violate the following business rules in order to allow them to resolve business key collisions prior to performing a merge:
o Cease last Identifier
o Cease Name of default type
o Cease Status Record
o Cease Demographic Record
• There will be no “unmerge” functionality. Unmerging will be accomplished by manual reactivation and ceasing of IPCs and CPNs, which will only be available to Registry Administrators.
The requirement to unmerge is supported by providing a search for Registry Administrators that allows them to find providers that have no active CPN. This is needed so that the registry administrator can locate providers to unmerge.
1.1.1.4 Software changes
The software changes that must be made are:
• Modify provider_details_tree.jsp to improve clarity of presentation of material.
• Modify MaintainPotentialDuplicates to implement the three merge scenarios.
• The database schema must be modified to support a provider having many CPNs and IPCs.
• Provider query functionality must be modified to follow forward references (from a duplicate provider to the new retained provider).
• Modify MaintainPotentialDuplicates method maintainDuplicateEntries to implement merge and unmerge operations.
• Modify MaintainProviderBean to allow suspension of regular business rules for Registry Administrators attempting to perform a merge.
1.1.1.5 Business and Technical Impacts
There are no business or technical impacts to existing PRS functions or data.