tommy's scratchpad

We just released beta1 yesterday. And lots of new features (enhanced templating, SQL Server 2008 support, Google API support for tiling) and bug fixes. Oh, and the OGC support grows; WCS 1.1, WFS 1.1, SOS 1.0.0 to name a few new implementations. While full CITE compliance is not quite complete for some of these, I think the initial implementations are very solid, and have already passed many of the CITE assertions.

For me, I’m happy to see the increased OGC support, as well as broadening MapServer’s capabilities for defacto standards like Google tiling, as well as the new possibilities as a result of the templating (like GeoJSON, etc.). Community feedback on this release would be great, if you want to follow the release plan. Final release is slated for 09 July 2008.

What features are you looking forward to using for 5.2.0?

Good news.  With all the new features and performance improvements, I’m guessing I’m not the only one looking forward to this release.

Neat post by Matthew Perry. I couldn’t help but run the same command:

$ history|awk '{a[$2]++ } END{for(i in a){print a[i] " " i}}'|sort -rn|head

188 cd

151 svn

113 vi

88 ll

60 sudo

53 more

40 make

40 ./dd.py

30 ./run_test.py

30 ls

dd.* is my tempfile name prefix for quick and dirty scripts. run_test.py is from running the MapServer msautotest regression test suite when developing and testing code.

What does your shell history look like?

I was compelled to write this after reading about Paul’s experiences. A couple of months ago, my Motorola E810 came to a screeching halt: the display died. As well, the battery recharge became spotty at best. So I was in need of a new phone. I called my wireless provider, who give me a story about having to lock into another 3 year contract to get a new phone.

I was very hesitant this time about locking in again; I’ve taken a beating on cellphone costs in the past, and I didn’t want to sign up for another “plan” again until I had reviewed all my options thoroughly. So I wasn’t given a phone. I asked if I could cancel my contract, which would have costed something ridiculous.

So at this point I was stuck. A friend lent me an old cellphone which I’ve been using in the meantime until I figure out what to do (another friend has since coined it the “loser phone” 🙂 ). So don’t laugh too hard next time you see me with a circa 2000 cellphone / brick on the table 🙂

What’s a guy to do? The prices of wireless communications are ridiculous! Is it like this everywhere (Man, my Vodafone was dirt cheap when I was last in Europe). Contract? Pay as you go?

I couldn’t agree with you more, Paul!

You may or may not have noticed that I’ve been quiet on the blogosphere, and MapServer to some extent. Well, it’s a result of kicking my basement renovation into high gear after starting last fall. And with the winter we’ve had, what better time for this project 🙂

I’ve done my share of renovations in the past, but, in terms of hackery, this one takes the cake. It’s fun to be hacker as a coder sometimes, but not when doing renovations! In tearing down the basement, I have undoubtedly seen the most horrific attempts at a basement reno in my entire life. I have never (and I mean never!) seen anything like this:

• using cardboard as shims, even in – get this – the shower stall. Now that won’t absorb any water!
• disaster electrical: connections put together with tape only (so much for junction boxes!). And behind the shower as well! When they did use boxes, there was no reason to. In cleaning up the electrical, we ended up pulling down over 10 junction boxes, which served no functional purpose whatsoever. Unbelievable!
• whoever taped and mudded the drywall did an awesome job of applying enough compound to make the joints resemble a beer belly and a half!

I could go on, but I won’t. I’ve often been called a perfectionist for renovations, but when you’re doing things that affect safety, it’s a big no no. Their should always be a great consideration for safety before proceeding with things. It could be as simple as making you do not accidentally step on tools or using a tool to lift drywalls easier. I think this property would have been worth more without any renovation than with the original attempt.

So after taking the whole thing down, I’ve been putting things back together again in the last 8 weeks. It’s been a fun experience for sure, but has also involved alot of running around, coordinating various help, playing helper and actually doing the work, and, of course, being budget conscious. Doing this at night after your day job is not for the faint at heart, but is gratifying in terms of cost as well as getting exactly what you want.

I hope to be done the bulk of the remaining work in the next month, which includes laying ~1000 square feet of porcelain/ceramic tile. Luckily, a friend’s uncle is helping with most of the project, so it’s valuable to have professional eyes (and hands) on things at all times. If you want to build a kitchen in your basement, you may need to work with a contractor who specializes in garbage disposal replacement in western new york to install a garbage disposal for you.

Check out the progress (advice/suggestions valued!).  When the drywall dust settles, I looking forward to getting my life back. Now that warmer temperatures are (seemingly) here, it’s high time to finish the job!

Between a dual boot and a MacBook, I have started to have bookmark synchronization issues.

I came across Foxmarks, a neat bookmark synchronizer for Firefox. Essentially, your bookmarks are placed on a central server so you can fetch them from wherever. The only change that I made was removing some internal / sensitive bookmarks which I wouldn’t necessarily want to be “out there”, so I stored those locally.

Pretty easy!

OGC service standards (such as WMS, WFS, WCS, SOS) support something called updateSequence. In short, this value represents an identifier which clients (catalogs, applications, etc.) can use to maintain their local copy up to date. This saves clients from fetching and parsing an entire (sometimes hefty) Capabilities XML document in cases where the server has not made any updates.

Thanks to a recent enhancement, MapServer now supports updateSequence for WMS Server, WFS Server, WCS Server, and SOS Server. Support is triggered by mapfile metadata MAP/WEB/METADATA/ows_updatesequence. The value can be a string, integer or ISO8601 timestamp.

We initially thought of using the mtime of the mapfile, but this may not have been the most indicative. What if other files (pointed to by the mapfile) are modified, and not the mapfile? What if the mapfile is touch’d and and no content changes are made? For this reason, we went with a user-specified value. Basically, if the user knows the purpose updateSequence, they can use it and manage it at their own discretion.

I suppose the OGC could have allowed using the HTTP Last-Modified header, however that may get slippery in terms of semantics.

Give it a shot! I’d be interested in hearing how this works with existing catalogues and applications. The enhancement seems to be passing the CITE tests initially (let’s keep our fingers crossed!). At any rate, it is fun to code an enhancement like this. It’s also nice to have dedicated tests to all the use cases surrounding updateSequence within msautotest/wxs/.

Some of the MapServer developers may have been getting more than their share of trac notifications this week. This is partly because I’ve taken the last few days to sift through many tickets which have no milestone attached to them. Between old issues, bugs that have since been fixed, etc., it was nice to clean house a bit. They contributed to ~20 tickets being closed just this week! It’s amazing to see how many old school tickets there are lying around in trac. I try to commit to a once a month sift of trac for the oldies, but sometimes goodies 🙂

Read the rest of this entry »

That’s how I take my coffee 🙂 You may have heard of Tim Hortons — a mecca for coffee shops in Canada. Many of us are lost without it during our morning commute. Even though they are seemingly everywhere, there are times when you just can’t find one.

A friend of mine pointed me to a forum working on collecting locations of all Tim Hortons locations in Canada and the US. Wouldn’t it be great to upload these to a GPS which supports GPX format? Or to visualize in Google Earth, for example?

This is easily done using open source geospatial tools. OGR supports the reading and writing of a number of formats, including GPX, KML, GeoJSON, Shapefile, and so on. Here’s how:

• Download http://www.gpspassion.com/upload/team/Dewi_TimHortons_xls_20071120.zip th.zip
• Save as CSV (there is a CSV online, but the XLS has more detailed description of each location)
• Use OGR’s Virtual Datasource support to read the CSV. Here’s the OVF definition:

<OGRVRTDataSource>
<OGRVRTLayer name="th">
<SrcDataSource relativeToVRT="1">./th.csv</SrcDataSource>
<GeometryType>wkbPoint</GeometryType>
<LayerSRS>WGS84</LayerSRS>
<GeometryField encoding="PointFromColumns" x="Longitude" y="Latitude"/>
</OGRVRTLayer>
</OGRVRTDataSource>

Transform to your heart’s content:

$ ogr2ogr -f KML ./th.kml ./th.ovf

$ ogr2ogr -dsco “GPX_USE_EXTENSIONS=YES” -f GPX ./th.gpx ./th.ovf

I’ve bundled up a zipfile of various formats for use as an example — have fun!

Since I did this last year, I thought I’d try this again for 2007. Here’s a lowdown for my 2007:

• REST is really here: and as popular as ever. For me, I finally realized that REST was a style, not a syntax, API or schema for that matter. Sean’s recent post sums it up quite nicely. It’s nice to see the OGC has acknowledged this
• Mass Market is really here: Things like Google, Yahoo Maps, and lowering the barrier to application development and, more ultimately, content, are paramount in the Web 2.0 sphere of things. Defacto standards / approaches matter
• Geospatial Catalogues: I blogged about this last year. Even with Cat 2.0 CSW / ebRIM formally approve, where is catalogue interoperability? Is a federated / distributed catalogue approach realistic in the near future?
• MapServer:
  • SOS Server: There has been much development here, including forthcoming support for 1.0.0, as well as implementation of POST support in mapogcsos.c. SWE DataBlock has also been implemented. We’ve also introduced automated testing for SOS in msautotest/wxs/sos.map. And a slew of bug fixes, memory leak fixes, etc. So I’m really happy about how SOS Server support has progressed this year. It goes without saying that major props to Assefa are in order here, as well Charlton and other MapServer SOS Server testers who have provided valuable feedback
  • OWS Common: Through increased usage in SOS Server, as well as the WCS 1.1 Server support (thanks Frank!), mapowscommon.c usage has been slowly increasing in the codebase to reuse existing functionality. This is a result of the OGC standards gradually adopting OWS Common for their “common” bits of XML and such. mapowscommon.c is also becoming more robust as a result of more integration and testing. I’m also happy about the increased libxml2 support in MapServer. We used libxml2’s xpath functionality in supporting SWE DataBlock, and have generic utilities now defined in maplibxml2.c
  • I’m happy to have become more involved in the MapServer codebase and project overall as part of the Project Steering Committee. There is definitely alot going on and alot to do, and the enthusiasm, commitment and helpful nature of the developers is great. MapServer is a special project to me, starting in early 2000 as part of my research / dissertation. The rest, as they say, is history. Oh, and “STYLES=” is optional again 🙂
• Publications: I was glad to see “The Geospatial Web” published this year. Nice to see new folks and goings on out there
• Adventures in Python: I decided to dive into python finally. With some sound advice from Sean, I embraced stuff like Genshi and etree. I managed to stand up a generic SensorML generator which I’m happy with, as far as my first Python project goes. My next goals are to experiement with doing things differently (like SQLAlchemy; I had to use good old SQL scripting as I ran out of time)

In other news:

• Construction time continues: I did some major work on my place, including new fencing, landscaping/gardening, central air conditioning, as well as a shiny new veranda. A bit more curb appeal for sure
• Condo: I finally picked up my condo this year. After much thought, I decided to sell. This was a beautiful property in a can’t lose location! Onwards
• Lifestyle changes: Perhaps my biggest victory is quitting smoking. I have been absolutely smoke free for all of 2007. I can’t say that there haven’t been times where I wanted one from time to time, but I feel much healthier and better overall

So as 2008 quickly approaches, here are a few things I’m looking forward to:

• MapServer: I look forward to continuing in helping more with OGC support, as well as become more familiar with the inner workings of the codebase.
• OpenLayers: I really want to get into OpenLayers this year. I have not done much on client side work since the initial Mapbuilder days
• New Springer publication: keep an eye out for a new, exciting book on GIS and open source this year
• OWSContext and KML: it will be useful to see a resolution or unification of some sort between these two standards
• OpenSearch: I think OpenSearch, with the geo extensions, will make a big hit this year. Nothing like a simple search facility which is already supported in browsers
• I’m renovating my basement this year. Gutting the entire basement and installing a new bathroom and kitchen. I hope to have this done by summer 2008 — wish me luck!

So that’s it from here. I wish you and your loved ones the very best in the holiday season and for 2008!