Hey I searched around a bit and I found a pretty powerful library which offers what you are looking for more or less straight forward and even actively supports Unity types:

geometry3Sharp

The implementation into a Unity Project is as simple as

• Download the project as .zip
• unpack the folder geometry3Sharp-master into your Assets folder

• In Unity under ProjectSettings → Player → Other Settings → Configuration → Scripting Define Symbols insert

  G3_USING_UNITY;
  

  Just as also explained in the README:

  geometry3Sharp supports transparent conversion with Unity types. To enable this, define G3_USING_UNITY in your Unity project, by adding this string to the Scripting Define Symbols box in the Player Settings.

Then you can simply calculate the edges of an oriented bounding box for a given array of Vector3 points like this:

using UnityEngine;
using g3;

public class Example : MonoBehaviour
{
    // Just for the demo I used Transforms so I can simply move them around in the scene
    public Transform[] transforms;

    private void OnDrawGizmos()
    {
        // First wehave to convert the Unity Vector3 array
        // into the g3 type g3.Vector3d
        var points3d = new Vector3d[transforms.Length];
        for (var i = 0; i < transforms.Length; i++)
        {
            // Thanks to the g3 library implictely casted from UnityEngine.Vector3 to g3.Vector3d
            points3d[i] = transforms[i].position;
        }

        // BOOM MAGIC!!!
        var orientedBoundingBox = new ContOrientedBox3(points3d);

        // Now just convert the information back to Unity Vector3 positions and axis
        // Since g3.Vector3d uses doubles but Unity Vector3 uses floats
        // we have to explicitly cast to Vector3
        var center = (Vector3)orientedBoundingBox.Box.Center;

        var axisX = (Vector3)orientedBoundingBox.Box.AxisX;
        var axisY = (Vector3)orientedBoundingBox.Box.AxisY;
        var axisZ = (Vector3)orientedBoundingBox.Box.AxisZ;
        var extends = (Vector3)orientedBoundingBox.Box.Extent;

        // Now we can simply calculate our 8 vertices of the bounding box
        var A = center - extends.z * axisZ - extends.x * axisX - axisY * extends.y;
        var B = center - extends.z * axisZ + extends.x * axisX - axisY * extends.y;
        var C = center - extends.z * axisZ + extends.x * axisX + axisY * extends.y;
        var D = center - extends.z * axisZ - extends.x * axisX + axisY * extends.y;

        var E = center + extends.z * axisZ - extends.x * axisX - axisY * extends.y;
        var F = center + extends.z * axisZ + extends.x * axisX - axisY * extends.y;
        var G = center + extends.z * axisZ + extends.x * axisX + axisY * extends.y;
        var H = center + extends.z * axisZ - extends.x * axisX + axisY * extends.y;

        // And finally visualize it
        Gizmos.DrawLine(A, B);
        Gizmos.DrawLine(B, C);
        Gizmos.DrawLine(C, D);
        Gizmos.DrawLine(D, A);

        Gizmos.DrawLine(E, F);
        Gizmos.DrawLine(F, G);
        Gizmos.DrawLine(G, H);
        Gizmos.DrawLine(H, E);

        Gizmos.DrawLine(A, E);
        Gizmos.DrawLine(B, F);
        Gizmos.DrawLine(D, H);
        Gizmos.DrawLine(C, G);

        // And Here we ca just be amazed ;)
    }
}

And of course there is stuff like

orientedBoundingBox.Box.Contains(Vector3d)

for determine if a given point lies within that box.

Just because Ruzihm asked:

Ofcourse you can slightly change the upper script to simply use actual mesh vertices:

public MeshFilter[] meshFilters;

private void OnDrawGizmos()
{
    var vertices = new List<Vector3>();
    foreach (var meshFilter in meshFilters)
    {
        // have to multiply the vertices' positions
        // with the lossyScale and add it to the transform.position 
        vertices.AddRange(meshFilter.sharedMesh.vertices.Select(vertex => meshFilter.transform.position + Vector3.Scale(vertex, meshFilter.transform.lossyScale)));
    }

    var points3d = new Vector3d[vertices.Count];

    for (var i = 0; i < vertices.Count; i++)
    {
        points3d[i] = vertices[i];
    }

    // ...
    // From here the code is the same as above

Which looks basically the same

Don't think of the box and then try to adapt it. It's easier to generate the box in this case. So points first, box later.

1. Find the two points most distant from eachother.

2. These will be two opposite corners of the box.
3. Calculate the direction/normal to the opposite point.
4. Calculate offset directions from that that will point to the missing points.

5. Generate the rest of the points of the box. Basically the directions you calculated by where they intersect the perpendicular paralel of the opposite point.
6. Generate the faces of the box based on its vertices.

I've illustrated for 2D here, but the only difference to 3D is a third angle, different angle offsets, and more vertices/faces to deal with.

EDIT:

As pointed out in the comments, this might not be a perfect solution. In cases where one or both extremities have multiple points that are close to, or at equal distance, from the opposing extremity, but separated laterally, this method will give you a decent approximation of the smallest bounding box, but will not give the smallest boudning box possible.

Ruzihm's illustration:

However, his illustration is a bit wrong, in that my solution is not axis-aligned. Here is what it would achieve, on the right, in red or orange, depending on which point is picked as the vertex:

Better/Correct solution:

While looking for a solution that would give the mathematically perfect minimum bounding box, I've found this other (C++) question and it's answers. Right now I don't have the time to read through it all and replicate the solution here for Unity/C#, so I will only point to it. Maybe I or another contributor can edit-in a replication of that later.

I was able to solve this by using OpenCV for Unity. I used the minAreaRect method, which calculates a fit bounding-box around the points in 2D (I projected them onto the X/Z plane first).

This method conveniently returned a rectangle, with a center point, angle and width / height. From here on it was quite easy to extend this to a 3D box.