Ways to improve performance for populating a material explosion tree using multiple micro services

Viewed 52

I want to do a material explosion. This means that given a product, I want to populate in a tree that has all the sub-components that are required to produce that product. Let's say to produce a table, I need 1 tabletop, 4 legs, 8 screws and 1 can of paint. Therefore, once I have this information, I can understand how much of these sub-components will be required to build 10 tables, by multiplying 10 with quantities required to build 1 table.

So given a material, I first populate it as the root node then I recursively fetch sub-components and start building the tree. My limitation is that when I fetch this information from another service I can only get immediate children, which means children at a single level.

Structure of my N-array tree looks like this:

public class Node<T> {

    private T data = null;
    private Node<T> parent = null;
    private List<Node<T>> children = new ArrayList<Node<T>>();

    public Node(T data) {
        this.data = data;
    }

    public Node(T data, Node<T> parent) {
        this.data = data;
        this.parent = parent;
    }

    public List<Node<T>> getChildren() {
        return children;
    }

    public Node<T> getParent() {
        return parent;
    }

    public void setParent(Node<T> parent) {
        this.parent = parent;
    }

    public void addChild(T data) {
        Node<T> child = new Node<T>(data);
        this.addChild(child);
    }

    public void addChild(Node<T> child) {
        child.setParent(this);
        this.children.add(child);
    }

    public void addChildren(List<Node<T>> children) {
        for (Node t : children) {
            t.setParent(this);
        }
        this.children.addAll(children);
    }

    public T getData() {
        return this.data;
    }

    public void setData(T data) {
        this.data = data;
    }

    public boolean isRoot() {
        return (this.parent == null);
    }

    public boolean isLeaf() {
        return this.children.size() == 0;
    }

    public void removeParent() {
        this.parent = null;
    }

    private class InOrderIterator implements Iterator<T> {

        private final Queue<Node<T>> queue = new LinkedList<Node<T>>();

        public InOrderIterator(Node<T> tree) {
            queue.add(tree);
        }

        @Override
        public boolean hasNext() {
            return !queue.isEmpty();
        }

        @Override
        public T next() {
            Node<T> node = queue.remove();
            queue.addAll(node.children);
            return node.getData();
        }

        @Override
        public void remove() {
            throw new UnsupportedOperationException();
        }
    }

    public Iterable<T> inOrderTraversal = () -> (Iterator<T>) new InOrderIterator(Node.this);
}

Now I create the root node, here the max depth is the level of tree up to which I want to fetch sub-components and construct the tree. The default quantity is 1, and the root level is 0.

protected Node<Material> doMaterialExplosion(final String partNumber, final Integer maxDepth) {
        Node<Material> root = new Node<>(Material.builder()
            .productPartNumber(partNumber)
            .materialPartNumber(partNumber)
            .materialQuantity(DEFAULT_QUANTITY)
            .level(ROOT_LEVEL)
            .parentPartNumber(null)
            .build());

        return doMaterialExplosion(root, partNumber, maxDepth);
    }

The product part number is the main product that has to be produced, material part number is the sub-component required to produce that product, material quantity is the quantity of that material required, the level is the tree level, and parent part number is the immediate parent of the subcomponent. So, for the root node, there will be no parent part number, but for all other sub-components, there will be a parent part number. Also, note that the product part number will not change in any node, as it specifies the main product that is being produced. The code snippet to construct this tree recursively is:

private Node<Material> doMaterialExplosion(Node<Material> root,
        final String partNumber, final Integer maxDepth) {

        // Increment the level, in the tree
        int childLevel = root.getData().getLevel() + 1;
        if (maxDepth != null && childLevel <= maxDepth) {
            // Fetch immediate sub-components, of a given material
            List<SubComponent> subComponents = fetchData(partNumber);

            Node<Material> child;
            if (!subComponents.isEmpty()) {
                LOG.info("'{}' child elements found for partNumber='{}'", subComponents.size(), partNumber);
                for (SubComponents subComponent : subComponents) {
                    child = new Node<>(Material.builder()
                        .productPartNumber(root.getData().getProductPartNumber())
                        .materialPartNumber(subComponent.getMaterial())
                        .materialQuantity(subComponent.getMaterialQuantity())
                        .level(childLevel)
                        .parentPartNumber(root.getData().getMaterialPartNumber())
                        .build(), root);

                    root.addChild(child);
                    doMaterialExplosion(child, subComponent.getMaterial(), maxDepth);
                }
            }
        }

        return root;
    }

The method fetch data, will give me a list of immediate subcomponents for the given part number, from another microservice.

What I am looking for is ways to improve my performance, because one can understand that, it can really get expensive if I want to create a tree that has many subcomponents, and each sub-component further has children and so on. Also, it will be expensive it the level of this tree construction is big. Let's say maybe up to level 10. And, also let's say based on some plan, I need to do this explosion for 50 products.

We also can't forget that this tree creation implementation is in one microservice, which gets data from another microservice. Therefore, for 50 products there can be a lot of network calls.

Limitation, we cannot have this implementation in the service where data is present. The biggest performance upgrade I am looking for is a way to reduce the time required to do material explosion for say N products.

At present, once I have this tree, I save it in a database, so that the next time I need a material explosion for the same product, I can get it much rapidly, this is also the reason for the structure of Material class, that has a product part number and parent part number.

Looking forward to your suggestions.

2 Answers

Data sharing is often difficult, but rou're right to try to keep the micro-service architecture here.

I can see 2 ways (and then a mix of the 2, perhaps):

  • Enhance the second micro-service API, so you can get a list of items from it, instead of requesting one by one (request machine gun!). It's becoming a common API feature... think of it as a search on the id, or in a GraphQL way.

  • Organize a cache on your first micro-service, like btilly suggested, and perhaps even more aggresive. A cache in memory according to your previous requests, or in a local database (nosql?) fed by the master micro-service.

The only thing your architecture lets you do to improve is that while fetch a product, you can memoize what you're already fetching. So if a table has 4 legs, you fetch a table once, a leg once, then apply the leg 4x. Then when you look for another table, you just look it up.

Related