Weird dimensions when trying to write a FITS BinaryTable (or AsciiTable) in Java

user_6552666 0
Viewed 27

This first function is stolen directly from the nom-tam-fits site: https://nom-tam-fits.github.io/nom-tam-fits/intro.html

private static void dataTableToBinaryFitsDummy() throws Exception {
    BufferedFile bf = new BufferedFile("table.fits", "rw");

     BasicHDU.getDummyHDU().write(bf);  // Write an initial null HDU

     double[] ra = {1.};
     double[] dec = {2.};
     String[] name = {"          "}; // maximum length will be 10 characters

     Object[] row = {ra, dec, name};
     long rowSize = ArrayFuncs.computeLSize(row);

     BinaryTable table = new BinaryTable();

     table.addRow(row);

     Header header = new Header();
     table.fillHeader(header);

     BinaryTableHDU bhdu = new BinaryTableHDU(header, table);

     bhdu.setColumnName(0, "ra", null);
     bhdu.setColumnName(1, "dec", null);
     bhdu.setColumnName(2, "name", null);

     bhdu.getHeader().setNaxis(2, 1000);  // set the header to the actual number of rows we write
     bhdu.getHeader().write(bf);

     ByteBuffer buffer = ByteBuffer.allocate((int) rowSize);

     for (int event = 0; event < 1000; event ++){
         buffer.clear();

         // update ra, dec and name here

         buffer.putDouble(event);
         buffer.putDouble(dec[0]);
         buffer.put(("event "+event).getBytes());

         buffer.flip();
         bf.write(buffer.array());
     }

     FitsUtil.pad(bf, rowSize * 1000);
     bf.close();
}

As a starting point, I want to be able to pass in an arbitrary collection of primitive double arrays. This was my attempt, which IMO shouldn't be particularly different from the original.

private static void dataTableToBinaryFitsDummyWithArgsDoublesOnlyWrapper() {
    int nRows = 100;
    
    double [] rowA = new double[nRows];
    double [] rowB = new double[nRows];
    double [] rowC = new double[nRows];
    double [] rowD = new double[nRows];
    
    Random random = new Random();
    for (int i=0;i<nRows;i++) {
        rowA[i] = i;
        rowB[i] = 2*i;
        rowC[i] = random.nextGaussian();
        rowD[i] = rowC[i] * -1;
    }
    
    ImmutableList<double[]> columns = ImmutableList.of(rowA, rowB, rowC, rowD);
    ImmutableList<String> names = ImmutableList.of("ints", "double ints", "randos", "neg randos");
    
    try {
        dataTableToBinaryFitsDummyWithArgsDoublesOnly(columns, names, 4, nRows);
    } catch (Exception e) {
        e.printStackTrace();
    }
    
}
private static void dataTableToBinaryFitsDummyWithArgsDoublesOnly(ImmutableList<double[]> columns,
        ImmutableList<String> names, int nCols, int nRows) throws Exception {
    BufferedFile bf = new BufferedFile("tableWithArgs.fits", "rw");

     BasicHDU.getDummyHDU().write(bf);  // Write an initial null HDU
     Object[] row = new Object[nCols];
     for (int i=0;i<nCols;i++) {
         row[i] = columns.get(i);
     }
     long rowSize = ArrayFuncs.computeLSize(row);
     
     BinaryTable table = new BinaryTable();
     table.addRow(row);
     
     Header header = new Header();
     table.fillHeader(header);
     
     BinaryTableHDU bhdu = new BinaryTableHDU(header, table);
     
     for (int i=0;i<nCols;i++) {
         bhdu.setColumnName(i, names.get(i), null);
     }
     
     bhdu.getHeader().setNaxis(2, nRows);
     bhdu.getHeader().write(bf);
     
     ByteBuffer buffer = ByteBuffer.allocate((int) rowSize);

     for (int i = 0; i < nRows; i ++){
         buffer.clear();

         // update ra, dec and name here

         for (double[] column : columns) {
             buffer.putDouble(column[i]);
         }
         
         buffer.flip();
         bf.write(buffer.array());
     }

     FitsUtil.pad(bf, rowSize * nRows);
     bf.close();
}

However, what I get looks very different. I want the rows to look like

ints, double ints, randos, neg randos
0.0, 0.0, 0.66938, -0.66938
1.0, 2.0, 0.53482, -0.53482
2.0, 4.0, 0.66825, -0.66825
...

But instead I get

ints, double ints, randos, neg randos
(0.0, 0.0, 0.66938, -0.66938, ...), (0.0, 0.0, 0.0, 0.0, ...), (0.0, 0.0, 0.0, 0.0, ...), (0.0, 0.0, 0.0, 0.0, ...)
(1.0, 2.0, 0.53482, -0.53482, ...), (0.0, 0.0, 0.0, 0.0, ...), (0.0, 0.0, 0.0, 0.0, ...), (0.0, 0.0, 0.0, 0.0, ...)

It looks like instead of getting 4 columns and 100 rows of numbers, I'm getting 4 columns and 100 rows of 100-number lists. I followed the canonical example as best I could, why did this happen?

While I'm at it, I also want this to be an AsciiTable and AsciiTableHDU, but the only examples I'm seeing are with BinaryTables, BinaryTableHDUs and ByteBuffers.

fits java
1 Answers

I'm not super familiar with nom.tam.fits, but according to the docs BinaryTable.addRow takes an array of arrays of primitives (this is because the data in a single cell of a FITS binary table can itself be a multi-dimensional array).

That's why the original example you cited creates an row from an array of 1-element arrays:

     double[] ra = {1.};
     double[] dec = {2.};
     String[] name = {"          "}; // maximum length will be 10 characters

     Object[] row = {ra, dec, name};

In your code you seem to be confusing rows and columns:

    double [] rowA = new double[nRows];
    double [] rowB = new double[nRows];
    double [] rowC = new double[nRows];
    double [] rowD = new double[nRows];

Here, each of these arrays represents a full column of data (which is why their lengths are nRows).

Then here

     Object[] row = new Object[nCols];
     for (int i=0;i<nCols;i++) {
         row[i] = columns.get(i);
     }

you create a single "row" containing for the values in that row the entire columns. You are missing some outer for-loop over nRows.

There is also a BinaryTable.addColumn which should probably more efficient for adding entire columns at once (especially if you pre-specify the number and types of columns).

The main use case of the original example, where one row is being written at a time, might be if you have some long-running observation or event logging process where one row is appended to the table at a time in a streaming fashion.

user_9822570
Related