Confusion reagrding Java HashMap collision

Viewed 435

I have custom class named Department, in which equals and hashCode are both overridden. Please find the snippet as below:

class Department {
    private final int id;
    private final String name;
    private final int count;

    public Department(int id, String name, int count) {
        super();
        this.id = id;
        this.name = name;
        this.count = count;
    }

    @Override
    public boolean equals(Object obj) {
        if (obj == null)
            return false;
        if (!(obj instanceof Department))
            return false;

        final Department emp = (Department) obj;

        return emp.name != null && emp.name.equals(name) && emp.count == count && emp.id == id;
    }

    @Override
    public int hashCode() {
        return count + name.length();
    }

    @Override
    public String toString() {
        return "ID: " + id + ", Name: " + name + ", Age: " + count + ", hashCode: " + hashCode();
    }
}

In the main method, I have initialized two departments in such a way that, their equals will return false but will have same hashcode. Those two departments are then added to a HashMap. Please find main method call as below:

public static void main(String[] args) {
        final Department dep1 = new Department(1, "software", 35);
        final Department dep2 = new Department(2, "software", 35);
        System.out.println("\n\nIs dep1.equals(dep2)? -- " + dep1.equals(dep2));
        System.out.println("Is dep1==dep2? -- " + (dep1 == dep2));

        System.out.println("\n\nDepartment 1: " + dep1);
        System.out.println("Department 2: " + dep2);

        final HashMap<Department, String> departmentHashMap = new HashMap<>();
        departmentHashMap.put(dep1, "Software 1");
        System.out.println("\n\nDepartment 1 added to map");
        System.out.println("Is Department 2 available in map? -- " + departmentHashMap.get(dep2));
        System.out.println("Is Department 2 key available in map? -- " + departmentHashMap.containsKey(dep2));
        departmentHashMap.put(dep2, "Software 2");

        System.out.println("\n\nDepartment 1: " + departmentHashMap.get(dep1));
        System.out.println("Department 2: " + departmentHashMap.get(dep2));

        for (final Entry<Department, String> entry : departmentHashMap.entrySet()) {
            System.out.println("Key: " + entry.getKey() + ", Value: " + entry.getValue());
        }
    }

As per the documents, when two different entries having same hashcode but not satisfying equals comparison, will cause collision in HashMap and entries will be stored as linked list. I did not observe this particular behavior. But when I iterated over the HashMap entries, they were fetched as individual entries, not linked list. Please find the output as below:

Is dep1.equals(dep2)? -- false
Is dep1==dep2? -- false


Department 1: ID: 1, Name: software, Age: 35, hashCode: 43
Department 2: ID: 2, Name: software, Age: 35, hashCode: 43


Department 1 added to map
Is Department 2 available in map? -- null
Is Department 2 key available in map? -- false


Department 1: Software 1
Department 2: Software 2
Key: ID: 1, Name: software, Age: 35, hashCode: 43, Value: Software 1
Key: ID: 2, Name: software, Age: 35, hashCode: 43, Value: Software 2

I could not reference exemplifying this particular case anywhere. Any help to clarify the concept will be highly appreciated.

6 Answers

I'll try to take you to the deep-level journey of Associative Array ADT, implementation of which is Data Structure in question - HashMap / HashTable.

I'll try to give some academic and theoretical background clear enough, so that you have better grasp of this topic.

HashMap is one implementation of the Associative Array Abstract Data Type (ADT), and this ADT is most frequently implemented as Hash Table data structure. So, you can think of HashMap and HashTable as conceptually same data structures, especially in Java, where only minor to the DS characteristics' level implementation (like thread safety, concurrency, ordering, etc.) differ.

In Hash Table (and also in the HashMap, I'll be hereinafter using these two structure names interchangeably), the most important feature of the data structure is that it gives you Ө(1) time for read, insertion, and update operations, by implementing associative data structure internally, and thanks to Hashing Function H(x) idea.

Hash Function is a fundamental concept in the Hash Table. It gets calculated and then normalized by Index Normalization in the underlying implementation.

Hash Table, under the hood, is implemented by its backing array. That backing array stores (is of type) either:

  1. Actual entries of the Hash Table, and hence, that backing array is of a type of HashTable’s specific entry type – Entry<K, V>[]. (Usually, Entry of the Hash Table is a special type/class, which holds that key and that value composition – i.e. which represents an Entry, and instances of which are maintained in the backing array; or
  2. Buckets of the entries of a Hash Table. Now, pay a close attention here, as I'm explaining this in a quite deep level. In this case, array would be of a type of Bucket, and each bucket, in turn, is going to be an instance of auxiliary data structure, which is usually LinkedList. So, long story short - in this case, you can imagine the backing array, that it will be something like LinkedList<K, V>[]. <- Each element of this array will be LinkedList instance, and in that instance you may have many objects.

Now, we're ready to introduce collisions.



Collisions

One of the important property of Hash Function H(x) is, that it must be Deterministic and Uniformal. A good uniformal H(x) gives you way less probabilities of collision - meaning it's very less likely that H(x) will hash two distinct inputs to the same output, however, this might happen! and for the two different inputs, you might get same output, which will get normalized to the same number, and effectively will point to the same slot of the backing array.

So, that's a Collision - when two input hash to the same index.

Q: How to handle this? A: There are two Technical Strategies to tackle this problem.

  1. Separate Chaining
  2. Open Addressing

Since your question addresses to backing array which stores List implementation, it's a Separate Chaining strategy, and I'll tell you few words on this (if you'll find my answer useful, I might later add explanation of Linear Probing as well).



Separate Chaining

Separate Chaining – deals with collisions by maintaining auxiliary data structure (usually Linked List, but other data structures may be used) to hold all the collisions, which are all those different keys which hashed to the same particular hash value. (Auxiliary data structure which holds collided keys, is sometimes called Bucket to represent the collection of many elements)

In this strategy/technique, as I've said above, each element of the backing array is Linked List (of Hash Table Entries) data structure, and whenever two or more elements (keys) collide (hashing to the same hash value), their entries are just added into the corresponding Linked List (which is placed in the position of collided hash values), but only if those entries have original (before hashing) keys different. If two entries’ keys collide after hashing, and those entries’ original keys are also equal, then the existing entry is replaced by the one we’re adding. If, say, Hash Table contains {3, "Subhrat"} entry and we’re adding one more entry {5, “David”}, but due to poor hashing function, 3 and 5 hashed into same value x, then the latter element will be just added to the corresponding Linked List (at index x of the backing array); however, if two keys hash to the same value and they also are equal in their original state (before hashing), then the existing entry will be replaced by latter.

Now comes the part which you didn't observe.

Q: How the Lookup is done in the case of Separate Chaining?
A:

  1. We give the key to the Hash Table;
  2. Key is hashed and resulted value represents the index of the backing array;
  3. 2nd step's corresponding slot in the array has a bucket (in our case – Linked List) and in that bucket original key (1st step) is looked-up/searched.

I hope this sheds some light on how Hash Map and Hash Table work, and now you understand more why you can't really see LinkedList fetched out.

The example you created is good. Internally there will be one entry in the hash map, and it's a linked list. However, there is no way of checking from the outside, meaning by using the Map API, if an entry is a linked list. The contract for Map and its iterators says it will deliver all items, individually and not in a specific order.
Have a look at the Java source code to see how the iterator works internally.

From your implementation dept1 and dept2 will be maintained as a linkedlist or (a possible TreeMap from JDK8) in same bucket in the HashMap . The reason dept1,dept2 will go to the same bucket is because they have the same hashCode() . So there will be collision.

From your ask , you won't be able to check the internals of the HashMap as how the elements are stored either a linkedlist or a TreeMap? because there are no public API's exposed and rightly so .That would be a leaky abstraction.

At a very high level the Map.entrySet() iterator scans the HashMap from bucket 0 , scanning the linkedlist (or a TreeMap) at each bucket and recursively doing the same for each and every bucket thus iterating every entries without telling us their internal structure

Why Equals() gets false? because you compare every attribute and the ids are

different so the output is false

Why I didn't get LinkedList while loop over entries? when you loop you use EntryIterator which reads node by node

, If you want to see the LinkedList you can use Mockito

package com.example;

import java.util.HashMap;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.mockito.internal.util.reflection.Whitebox;
import org.mockito.runners.MockitoJUnitRunner;

@RunWith(MockitoJUnitRunner.class)
public class ExampleClassTest {

    static class Department {

        private final int id;
        private final String name;
        private final int count;

        public Department(int id, String name, int count) {
            super();
            this.id = id;
            this.name = name;
            this.count = count;
        }

        @Override
        public boolean equals(Object obj) {
            if (obj == null) {
                return false;
            }
            if (!(obj instanceof Department)) {
                return false;
            }

            final Department emp = (Department) obj;

            return emp.name != null && emp.name.equals(name) && emp.count == count && emp.id == id;
        }

        @Override
        public int hashCode() {
            return count + name.length();
        }

        @Override
        public String toString() {
            return "ID: " + id + ", Name: " + name + ", Age: " + count + ", hashCode: " + hashCode();
        }
    }

    @Test
    public void shouldPrintCollision() {

        final Department dep1 = new Department(1, "software", 35);
        final Department dep2 = new Department(2, "software", 35);

        final HashMap<Department, String> departmentHashMap = new HashMap<>();
        departmentHashMap.put(dep1, "Software 1");
        departmentHashMap.put(dep2, "Software 2");

        Object[] array = (Object[]) Whitebox.getInternalState(departmentHashMap, "table");
        Object firstNode = null;
        for (Object obj : array) {
            if (obj != null) {
                firstNode = obj;
            }
        }

        printRecusive(firstNode);
    }

    private void printRecusive(Object node) {
        if (node == null) {
            return;
        }
        System.out.println(node);
        Object next = Whitebox.getInternalState(node, "next");
        printRecusive(next);
    }
}

, output

ID: 1, Name: software, Age: 35, hashCode: 43=Software 1
ID: 2, Name: software, Age: 35, hashCode: 43=Software 2

At the academic level, hash containers can deal with collisions a myriad of ways, but basically the bucket can be either a pointer/reference to a single item or to some sort of secondary container. Both flavors have their advantages and costs.

  • If it is a secondary container, all the hits go in there for whatever kind of search that container supports; containers can be created on the first bucket add.
  • If a pointer/reference and not null, hits have to go in other empty buckets selected by a specific sequence: linear, quadratic, double hash, and you can Google for many more -- smells like a popular dissertation topic. With pointer/reference hash containers, on miss search continues until a null bucket is found.

JAVA uses a linked list secondary container. Since hash containers are not ordered, the secondary container order is not important, so that is a sensible choice: cheap to add and linear to search. An iterator also need not worry about the order out of a hash map, as long as every element pair is iterated.

The smart thing with a hash container of any sort is to set the initial size really big, large enough so collisions are rare. An empty bucket is a pointer/reference, 8 bytes, but grows by the overhead of the secondary container for each add of those bucket types, so it is a basic space vs. speed trade-off! I speculate that non-mod-2 sizes might be better, although mod-2 sizes might divide quicker (an and would do it), and prime numbers seem especially nice for randomizing bucket choice.

It should go without saying that the hash function should also be as random as possible.

Some hash containers can be doubled in size, but then, after the bucket list is duplicated for twice as many buckets, half the items are in the wrong bucket, so no free lunch. Until it is cleaned, on iteration all bucket items have to have their hash checked, on find there will be more bucket searching, and perhaps impromptu cleaning on add. JAVA does not seem to have expandable hash containers.

Please consider when we are talking about programming there are two different concepts, implementation and abstraction. In this case when you are talking about LinkedList in the Hashmap this LinkedList is used in the internal imlementation of Hashmap, this means internally when HashMap receive two keys with the same hashcode it stores those entries (with the same hashcode) in the same LinkedList but you can't see this internal implementation as the user of this api unless you go through the code inside HashMap which has implemented this behavior.

On the other hand when you are testing HashMap in your main mehod you are actually testing external representation of the HashMap api which is exactly every HashMap user expects. They expect when they put an element in the HashMap with a key then in the future they can request HashMap to get that element with the same key(same here means two keys that are equal) notice that the hashcode of the key is not important for the user of the HashMap( this sentence is only correct in terms of functionality and not performance). The only rule you should know as a user of HashMap is that when two keys are equal they must have the same hashCode.

hashCode here is used in terms of performance and not functionality. Imagine your hashCode always return fixed integer( for instance 1) for all instance of Department even in this case HashMap works fine. But in this case all your element stored in one list in internal implementation which is very slow. For making this more complicated you can consider String and Object as keys in HashMap.

But why HashMap uses LinkedList in its internal implementation? to make long story short when we are talking in terms of data structure array are good for ramdom access but they need lots of memory. Assume that your key is of integer type, you can use an array to store element but this way you should have an array of lentgh 2147483647(lets put away negative number in this case) but this way you can access your emenet by key in (O1). Another approach is using LinkedList this way you should store your key with value in an entry of LinkedList. This way you have very little memory usage because you allocate memory (when new entry arrive) at the end of your LinkedList; however, the downside of this approach is its performance since when you want to find element by key you should iterate through all of your element in the LinkedList which is very slow. HashMap implementation has done its best to have the best of both world by mixing array and LinkedList.

In a nutshell this implementation has reduced the size of array that is needed using hashCode. It tries to dispatch element in its internal array using hashcode and allow different keys to have the same hashcode, so this way we don't need to have an array with the size of all possible value for key. so with this implementation we can have smaller size array. but in case of collision(when two keys have the same hashCode) they are stored in the same entry of array and actually each entry of array is a linkedList. When we request an element from Hashmap providing it with the key, hashmap frist find array entry by computing hashcode for that key and find the entry (which is actually a linkedList) then iterate through that LinkedList and compute equal for each element of that LinkedList until it find element which is equal to provided key. this way we have performance and small memory allocation together.

Related