Find duplicate records in MySQL

The key is to rewrite this query so that it can be used as a subquery.

SELECT firstname, 
   lastname, 
   list.address 
FROM list
   INNER JOIN (SELECT address
               FROM   list
               GROUP  BY address
               HAVING COUNT(id) > 1) dup
           ON list.address = dup.address;

SELECT date FROM logs group by date having count(*) >= 2

Why not just INNER JOIN the table with itself?

SELECT a.firstname, a.lastname, a.address
FROM list a
INNER JOIN list b ON a.address = b.address
WHERE a.id <> b.id

A DISTINCT is needed if the address could exist more than two times.

Not going to be very efficient, but it should work:

SELECT *
FROM list AS outer
WHERE (SELECT COUNT(*)
        FROM list AS inner
        WHERE inner.address = outer.address) > 1;

This will select duplicates in one table pass, no subqueries.

SELECT  *
FROM    (
        SELECT  ao.*, (@r := @r + 1) AS rn
        FROM    (
                SELECT  @_address := 'N'
                ) vars,
                (
                SELECT  *
                FROM
                        list a
                ORDER BY
                        address, id
                ) ao
        WHERE   CASE WHEN @_address <> address THEN @r := 0 ELSE 0 END IS NOT NULL
                AND (@_address := address ) IS NOT NULL
        ) aoo
WHERE   rn > 1

This query actially emulates ROW_NUMBER() present in Oracle and SQL Server

See the article in my blog for details:

• Analytic functions: SUM, AVG, ROW_NUMBER - emulating in MySQL.

 SELECT firstname, lastname, address FROM list
 WHERE 
 Address in 
 (SELECT address FROM list
 GROUP BY address
 HAVING count(*) > 1)

    SELECT *
    FROM (SELECT  address, COUNT(id) AS cnt
    FROM list
    GROUP BY address
    HAVING ( COUNT(id) > 1 ))

I use the following:

SELECT * FROM mytable
WHERE id IN (
  SELECT id FROM mytable
  GROUP BY column1, column2, column3
  HAVING count(*) > 1
)

SELECT id, count(*) as c  
 FROM 'list'
GROUP BY id HAVING c > 1

This will return you the id with the number of times that id is repeated, or nothing in which case you will not have repeated id.

Change the id in the group by (ex: address) and it will return the number of times an address is repeated identified by the first found id with that address.

SELECT id, count(*) as c  
 FROM 'list'
GROUP BY address HAVING c > 1

I hope it helps. Enjoy ;)

    Find duplicate Records:

    Suppose we have table : Student 
    student_id int
    student_name varchar
    Records:
    +------------+---------------------+
    | student_id | student_name        |
    +------------+---------------------+
    |        101 | usman               |
    |        101 | usman               |
    |        101 | usman               |
    |        102 | usmanyaqoob         |
    |        103 | muhammadusmanyaqoob |
    |        103 | muhammadusmanyaqoob |
    +------------+---------------------+

    Now we want to see duplicate records
    Use this query:


   select student_name,student_id ,count(*) c from student group by student_id,student_name having c>1;

+--------------------+------------+---+
| student_name        | student_id | c |
+---------------------+------------+---+
| usman               |        101 | 3 |
| muhammadusmanyaqoob |        103 | 2 |
+---------------------+------------+---+

To quickly see the duplicate rows you can run a single simple query

Here I am querying the table and listing all duplicate rows with same user_id, market_place and sku:

select user_id, market_place,sku, count(id)as totals from sku_analytics group by user_id, market_place,sku having count(id)>1;

To delete the duplicate row you have to decide which row you want to delete. Eg the one with lower id (usually older) or maybe some other date information. In my case I just want to delete the lower id since the newer id is latest information.

First double check if the right records will be deleted. Here I am selecting the record among duplicates which will be deleted (by unique id).

select a.user_id, a.market_place,a.sku from sku_analytics a inner join sku_analytics b where a.id< b.id and a.user_id= b.user_id and a.market_place= b.market_place and a.sku = b.sku;

Then I run the delete query to delete the dupes:

delete a from sku_analytics a inner join sku_analytics b where a.id< b.id and a.user_id= b.user_id and a.market_place= b.market_place and a.sku = b.sku;

Backup, Double check, verify, verify backup then execute.

SELECT * FROM bookings WHERE DATE(created_at) = '2022-01-11' AND code IN ( SELECT code FROM bookings GROUP BY code HAVING COUNT(code) > 1 ) ORDER BY id DESC

Most of the answers here don't cope with the case when you have MORE THAN ONE duplicate result and/or when you have MORE THAN ONE column to check for duplications. When you are in such case, you can use this query to get all duplicate ids:

SELECT address, email, COUNT(*) AS QUANTITY_DUPLICATES, GROUP_CONCAT(id) AS ID_DUPLICATES
    FROM list
    GROUP BY address, email
    HAVING COUNT(*)>1;

If you want to list every result as a single line, you need a more complex query. This is the one I found working:

CREATE TEMPORARY TABLE IF NOT EXISTS temptable AS (    
    SELECT GROUP_CONCAT(id) AS ID_DUPLICATES
    FROM list
    GROUP BY address, email
    HAVING COUNT(*)>1
); 
SELECT d.* 
    FROM list AS d, temptable AS t 
    WHERE FIND_IN_SET(d.id, t.ID_DUPLICATES) 
    ORDER BY d.id;

Would go with something like this:

SELECT  t1.firstname t1.lastname t1.address FROM list  t1
    INNER JOIN  list t2 
    WHERE 
        t1.id < t2.id AND 
        t1.address = t2.address;