I like this question :)
As the scope of the question is broad, I will also present you with a broad answer. That means that this is not a specific implementation but rather an explanation of the concepts needed for this.
Hit testing
You presented hit testing as a way to deal with this issue. I believe that this is not feasible in most cases, let me explain.
Iteration problem: "keep performing hit tests along the edge line" - maybe there is a good algorithm for doing this in a somewhat efficient fashion, however, if you think about it, you would have to perform a lot of checks to get results depending on the approach you take (the difficult question here is how you determine success for your algorithm, i.e. when it should stop searching).
Also note that "Hit testing requires layout to be up-to-date but does not require painting to be up-to-date.", which means that it is not intended to rely on painting in hitTest - I am also not aware of a way to perform hit tests on a canvas, so the idea of easily checking where the canvas painted might not actually be possible.
Parent data
The way I would approach this problem is using parent data, specifically BoxParentData.
Ideally, you would paint your nodes using render objects as well because that allows you to work with the parent data easily.
Before I go into a little bit of how it can be implemented, here is my idea:
- You have a render object container (your
MultiChildRenderObjectWidget) that can handle your nodes.
- The nodes will have
GraphContainerNodeParentData (example name).
- Each node paints based on a description of the shape. This description could be a
Path (you could use PathMetrics to evaluate that later) or something simpler if you can find a way to simplify e.g. the description of the rounded rectangle.
- The node sets that shape description as its parent data (variables in the
GraphContainerNodeParentData.
- The render object container will be able to read the
GraphContainerNodeParentData, which contains the information about the shape. Now, you will be able to go through your children during painting and read the parent data, where the shape description is stored → problem solved :)
Implementation
This is the way Stack et al. work. You can find the implementation of rendering for Stack in the framework:
Furthermore, I used an abstract way of dealing with parent data in my open source Flutter Clock submission. If you are interested in understanding parent data better, it could be helpful. The abstract multi child (container) render object can be found here.
Simplification
You might not need to go that deep (depending on what you are trying to achieve).
You can also set parent data using a ParentDataWidget and potentially combine that with simpler ways of composing your shapes.
For example, you could just use a ClipRRect or something with a specific border radius and pass that border radius to the parent data. With some math, you will always be able to find the correct edges for your shapes with variable border radii in your multi child render object paint method :)
Abstraction
If you do not need to handle abstract cases, i.e. in your case all kinds of different shapes (which could be implemented using the parent data shape description as I outlined), you could also just leave out all of this.
Imagine you always use the same border radius. Why would you worry about even passing parent data then? You could simply calcuate where the edges are based on the size when you have a fixed border radius or fixed shape.
So I want you to keep in mind that even though I proposed this abstract way of dealing with it (which is not difficult at all to work with when you understand but can be cumbersome to get into), you should find the simplest way of solving the problem for your specific case.
More abstraction is always possible - I could e.g. pour a lot of effort into something like this, creating an extremely abstract API that can handle shapes of any kind (using PathMetrics e.g.) to always find the perfect spots, no matter what kind of cubics you used to paint your nodes. However, that might be completely unnecessary and even lead you off track because you are not able to handle the more difficult solution.
Approach 1: abstraction for all cases
If you are looking for something abstract, look at my canvas_clock implementation for inspiration - it uses basically only RenderBoxes, so you will find what you are searching for in that :) In hindsight, the code quality is not amazing, the structure was not well chosen, and it obviously glosses over hit testing, intrinsic sizing, etc., however, for what it does, it goes the way of the abstract extreme (:
Approach 2: pragmatism for a specific case
There are a bunch of exisitng abstractions (like ParentDataWidget and CustomPainter) that can be used instead and you might not even need to handle different shapes (just a bit of math if you e.g. always draw the same rounded rectangle).
If you are only interested in one specific shape, I think that most of the parent data stuff is not strictly necessary :)
Conclusion
I think that I presented you with a few approaches for how this could be pulled off. I did not go into any specifics (maths or how to do it using PathMetrics - hint: you can use one Path object for canvas.drawPath and also extract information using PathMetrics), however, that is due to the broad nature of the question.
I hope that this information was useful to you in any way - I sure did enjoy sharing my thought :)
Btw, I am sorry for the ramble. I would consider this a low quality answer because I only quickly wrote down my thoughts instead of thoroughly structuring the answer and conducting some more research.
