Sunday, May 12, 2019

Man with no math degree finds fatal flaw in haversine formula

This story isn't very political, but it shows the value of old-fashioned common sense and tinkering.

The haversine formula is a very complicated formula that computes the great circle distance between two points on this fine planet. I've copied it for use in FreeBASIC programs, but I don't actually understand it. The highest level of math education I have is lower college level. I had to take calculus as a high school senior, but I don't remember any of it. In fact, I don't remember anything I learned in college math either. I don't remember anything I learned in math class after I was a high school sophomore, even though math was by far my best subject. We never studied the haversine formula.

Nobody today appreciates people like me who tinker with their work and learn from trial, error, and experience. Nowadays, you're expected to be a people person instead - which I'm not. But for over a decade, I've tinkered with FreeBASIC programs I wrote that attempt to link every county in the U.S. with a nearby urban area.

Now I've discovered something weird. When computing the distance between a certain point in Greene County, Illinois, and a point on the outskirts of nearby Springfield, the haversine formula gives a distance that's way off. It says it's about 36 miles when actually it's about 44 miles.

It's not a bug in the way FreeBASIC or my computer does trigonometric functions. I visited 2 different websites that compute great circle distances using this formula, and the same thing happened.

I notice that the line between these two points crosses the 90th meridian. I used this formula to figure out other distances in Illinois that also cross the 90th meridian, and those were also way off. The formula works out fine as long as the line doesn't cross the 90th meridian.

The Internet is full of techies and math geniuses. But I can't find any mention anywhere of this flaw in the haversine formula.

I just discovered something that the world's greatest mathematicians have never even discovered - even though it's staring us in the face!

2 comments:

  1. I'm not a math expert but somebody mentioned it somewhere I'm sure..

    ReplyDelete
    Replies
    1. There has to be a workaround. Otherwise, every airline flight that crosses the 90th meridian would be off by a significant amount.

      I'm trying it with other lines that cross the 90th meridian, and it works fine. I can't recreate where the other points were.

      Delete