How to view data inside IDataView

user_7441056 0
Viewed 1456

In ML.net, I want to view the data in the IDataView to verify whether the right data is loaded. I cannot see any visualizer/debugging tool to view them just like we view in System.Data.DataTable as a table

var mlContext = new MLContext();
IDataView trainData = mlContext.Data.LoadFromTextFile<TaxiTrip>(GetAbsolutePath("../../taxi-fare-test.csv"), hasHeader: true);
ml.net
3 Answers

There's a Preview method for the IDataView. You can run that method in a variable and debug through there to look at the data.

var preview = trainData.Preview();
user_1860130

What exactly I was looking for is to watch the data in a tabular format. In order to achieve this I,ve written an extension method for IDataView object below which will convert the data into a DataTable

Extension

public static class DataViewHelper
{
    public static DataTable ToDataTable(this IDataView dataView)
    {
        DataTable dt = null;
        if (dataView != null)
        {
            dt = new DataTable();
            var preview = dataView.Preview();
            dt.Columns.AddRange(preview.Schema.Select(x => new DataColumn(x.Name)).ToArray());
            foreach (var row in preview.RowView)
            {
                var r = dt.NewRow();
                foreach (var col in row.Values)
                {
                    r[col.Key] = col.Value;
                }
                dt.Rows.Add(r);

            }
        }
        return dt;
    }
}

Usage

var table= dataView.ToDataTable();

If you now debug the variable table, you could use the in-built DataTable Visualizer of visual studio to see the data

user_74410560

Snip,

public static void LogDataView(IDataView dataView)
{
    var preview = dataView.Preview();
    foreach (var col in preview.Schema) {
        if (col.Name == "Features") {
            continue;
        }
        Console.Write(col.Name + "\t");
    }
    foreach (var row in preview.RowView) {
        Console.WriteLine();
        foreach (var col in row.Values) {
            if (col.Key == "Features") {
                continue;
            }
            Console.Write($"{col.Value}\t");
        }
    }
    Console.WriteLine("\nCount: {0}\n", preview.RowView.Length);
}

Skip Features because it is a vector, not printable

user_17129130
Related