Export SimpleGeo Layers with Python

Urban Airship has decided to shut down the SimpleGeo services that it acquired by March 31st, 2012. I personally see this as step-backwards for the services that they offer and to the SimpleGeo customer’s who rely on the data.

* stiQRd (I am lead developer) uses it for all of our location information.

What was most shocking to me after their announcement wasn’t the cut-off date or the fact that Urban Airship had decided to sunset the service but the fact that they “punted” the solutions for helping existing customer’s migrate their data.  Granted they did offer semi-replacements for Storage, but it basically amounts to you rolling your own solution.  That is why we were using SimpleGeo in the first place...because we didn’t want to roll our own because of limited time and resources.  

SimpleGeo was so easy to use and the company had great support. It was a no brainer.

In order to prepare and update the stiQRd app before the sunset date I need to get an export of my data.  Unfortunately, there isn’t an easy way to export layer data programmaticly or through the console.  I posted a question to the Google Group and got some half-ass response that didn’t help whatsoever. Not to be derailed I was hacking together a few solutions, none of which I really liked.  I then came across this gem (I know...it is bad) of a Ruby script that will iterate through all of your layers and export that data to csv. Score!

The tips to retrieving all your data from a particular layer are two fold. SimpleGeo’s API doesn’t have a “get_all_records(layer_name)” type of method, but they do have “get_nearyby”, which at face value doesn’t look like it will give you the necessary results, but it does when you use (0,0) as the lat/lon and add in the “bbox” (boundary box) parameter values of ‘-90,-180,90,180’.

Unfortunately, I don’t know Ruby very well and I am spending a lot of my free time sharpening my Python skills. Since there wasn’t an implementation for export in Python I decided to port the Ruby script.  With the help of Bob “I am a PEP freak” Waycott, we have a pretty elegant solution.

I am also happy to repost that Parse has taken it a few steps further and offered up the simplest/elegant solution for migrating this data over to their backend service.  If you are looking for a turnkey solution then head over their for their migration tool.

If ANYONE has suggestions or updates please let me know.

Python Migration Script: (written by Cory D. Wiles and Bob Waycott)
2 responses
Geocoda has launched a geocoding and spatial database service that compares favorably to the former Context and Storage APIs. We also have a converter in Ruby that you can run to transform your data, and would be happy to work with you directly to get any raw export file you have loaded. Check us out at https://geocoda.com, and write support [-at-] if you have any questions.
C.G. thanks for sending me this. I'll definitely check it out.