We always in inevitability end up with duplicates data where we don’t need it. This is a simple way to find and remove those duplicates. I have documented this procedure for MSSQL 2008 up. Email me if you want the same type of procedure for Databases where you cannot run a CTE (common Table expression).
Finding the duplicates
The first hurdle to success is to find the columns that we can use to determine whether the duplicate exists, this is don’t by writing a query to verify the duplicate Data exists, without that then this is almost a pointless task. In my case It was with a customer Table (Table and Column obfuscated names obviously). Numerous customers had value had duplicate ProfileKey’s, whilst this is only an idea it was critical to avoid duplicates in many systems such as billing and customer management systems.