Sqlalchemy: Latitude and Longitude Float Precision?

user_693754 0
Viewed 9503

I'm using Sqlalchemy to define my tables and such and here is some code I came up with:

locations = Table('locations', Base.metadata,
Column("lat", Float(Precision=64), primary_key=True),
Column("lng", Float(Precision=64), primary_key=True),
)

I read somewhere that latitude and longitude require better precision than floats, usually double precision. So I set the precision manually to 64, is this sufficient? Overkill? Would this even help for my situation?

pylons geolocation sqlalchemy database-design database
4 Answers

TL;DR: if one-meter resolution is acceptable then a single-precision float storing degrees is acceptable.

This answer is a bit late to the party but I needed a solid answer myself and so hacked out some code to quickly get it. There are of course more elegant ways to do this, but it looks to work. As noted by Jeff, the worst case scenario will be at +/- 180 degrees longitude (ie, the date line).

Per the code below, a single-precision float is accurate to 0.85 meters at the date line using single-precision floats storing degrees. Accuracy increases significantly (to w/in mm) when close to the Prime meridian.

#include <stdio.h>

// per wikipedia, earth's circumference is 40,075.017 KM
#define METERS_PER_DEG  (40075017 / 360.0) 

// worst case scenario is near +/-180.0 (ie, the date line)    
#define LONGITUDE    180.0

int main()                                                                      
{                                                                               
   // subtract very small but increasingly larger values from
   //    180.0 and recast as float until it no longer equals 180.0
   double step = 1.0e-10;   
   int ctr = 1;
   while ((float) LONGITUDE == (float) (LONGITUDE - (double) ctr * step)) {
      ctr++;
   }
   double delta = (double) ctr * step;           
   printf("Longitude %f\n", LONGITUDE);
   printf("delta %f (%d steps)\n", delta, ctr);  
   printf("meters: %f\n", delta * METERS_PER_DEG);
   return 0;                                                                    
}

Output from this code is

Longitude 180.000000 
delta 0.000008 (76294 steps) 
meters: 0.849301
user_41392800
Related