Geocaching Log Feed Added

Since I am limited to Blogger for publishing our “My Finds” PQ, I’ve used a WordPress widget to add a link to the latest posts here.

You can find them on the Geocaching page.

Blog Your Geocaching Found Logs with Blogger

You may or may not be aware that Google have recently released a command line tool called Google CL which allows limited updating of some of it’s primary services from the command line – including Blogger.

I have been working on a script and looking for a utility to parse the “My Finds” pocket query for uploading to a blog for a while now so on hearing this news I set to work to see if I could create an automated script. You can see the results on my old blogger account, which I have now renamed _TeamFitz_ and repurposed for publishing our Geocaching adventures.

It’s a little bit clunky and could be improved, but the script is now complete and ready for ‘beta’. I’m publishing it here and releasing it under GPL for others to download, copy and modify for their own Geocaching blogs.

A few snags:

  • It will only work with one “Find” per cache – if you found twice it may screw up the parser.
  • Google have an arbitrary limit of 30-40 auto-posts per day, which is entirely fair, it will then turn on word verification which will prevent CL updates. I have limited the script to parse only 30 posts at a time.

You will need to download and install Google CL, it goes without saying the script is Linux only but if someone wants to adapt it to Windows they are welcome.

I have commented out the “google” upload line for test runs, remove # to make it active.

Either cut n’ paste the code below, or download the script from YourFileLink. Please comment and post links to your own blogs if you use it, also let me know if there are any bugs I haven’t addressed.

#!/bin/bash
# Script to unzip, parse and publish
# Pocket Queries from Geocaching.com to Blogger
# Created by Wes Fitzpatrick (http://wafitz.net)
# 30-Nov-2009. Please distribute freely under GPL.
#
# Change History
# ==============
# 24-07/2010 - Added integration with Blogger CL
#
# Notes
# =====
# Setup variables before use.
# Blogger has a limit on posts per day, if it
# exceeds this limit then word verification
# will be turned on. This script has been limited
# to parse 30 logs.
# Blogger does not accept date args from Google CL,
# consequently posts will be dated as current time.
#
# Bugs
# ====
# * Will break if more than one found log
# * Will break on undeclared "found" types
#   e.g. "Attended"
# * If the script breaks then all temporary files
#   will need to be deleted before rerunning:
#	.out
#	.tmp
#	new
#	all files in /export/pub
#####     Use entirely at your own risk!      #####
##### Do not run more than twice in 24 hours! #####
set -e
clear
PQDIR="/YOUR/PQ/ZIP/DIR"
PUBLISH="$PQDIR/export/pub"
EXPORT="$PQDIR/export"
EXCLUDES="$EXPORT/excludes.log"
GCLIST="gccodes.tmp"
LOGLIST="logs.tmp"
PQZIP=$1
PQ=`echo $PQZIP | cut -f1 -d.`
PQGPX="$PQ.gpx"
BLOG="YOUR BLOG TITLE"
USER="YOUR USER ID"
TAGS="Geocaching, Pocket Query, Found Logs"
COUNTER=30
if [ ! $PQZIP ];then
echo ""
echo "Please supply a PQ zip file!"
echo ""
exit 0
fi
if [ ! -f "$EXCLUDES" ];then
touch "$EXCLUDES"
fi
# Unzip Pocket Query
echo "Unzipping PQ..."
unzip $PQZIP
# Delete header tag
echo "		...Deleting Header"
sed -i '/My Finds Pocket Query/d' $PQGPX
sed -i 's/'"$(printf '\015')"'$//g' $PQGPX
# Create list of GC Codes for removing duplicates
echo "		...Creating list of GC Codes"
grep "<name>GC.*</name>" $PQGPX | perl -ne 'm/>([^<>]+?)<\// && print$1."\n"' >  $GCLIST
# Make individual gpx files
echo ""
echo "Splitting gpx file..."
echo "	New GC Codes:"
cat  $GCLIST | while read GCCODE; do
#Test if the GC code has already been published
if [ ! `egrep "$GCCODE$" "$EXCLUDES"` ]; then
if [ ! "$COUNTER" = "0" ]; then
echo "      	$GCCODE"
TMPFILE="$EXPORT/$GCCODE.tmp"
GCFILE="$EXPORT/$GCCODE"
sed -n "/<name>${GCCODE}<\/name>/,/<\/wpt>/p" "$PQGPX" >> "$TMPFILE"
grep "<groundspeak:log id=" "$TMPFILE" | cut -f2 -d'"' | sort | uniq > "$LOGLIST"
cat $LOGLIST | while read LOGID; do
sed -n "/<groundspeak:log id=\"$LOGID\">/,/<\/groundspeak:log>/p" "$TMPFILE" >> "$LOGID.out"
done
FOUNDIT=`egrep -H "<groundspeak:type>(Attended|Found it|Webcam Photo Taken)" *.out | cut -f1 -d: | sort | uniq`
mv $FOUNDIT " $GCFILE"
rm -f *.out
URLNAME=`grep "<urlname>.*</urlname>" "$TMPFILE" | perl -ne 'm/>([^<>]+?)<\// && print$1."\n"'`
echo "      	$URLNAME"
# Replace some of the XML tags in the temporary split file
echo "      		...Converting XML labels"
sed -i '/<groundspeak:short_description/,/groundspeak:short_description>/d' "$TMPFILE"
sed -i '/<groundspeak:long_description/,/groundspeak:long_description>/d' "$TMPFILE"
sed -i '/<groundspeak:encoded_hints/,/groundspeak:encoded_hints>/d' "$TMPFILE"
sed -i 's/<url>/<a href="/g' "$TMPFILE"
sed -i "s/<\/url>/\">$GCCODE<\/a>/g" "$TMPFILE"
LINK=`grep "http://www.geocaching.com/seek/" "$TMPFILE"`
OWNER=`grep "groundspeak:placed_by" "$TMPFILE" | cut -f2 -d">" | cut -f1 -d"<"`
TYPE=`grep "groundspeak:type" "$TMPFILE" | cut -f2 -d">" | cut -f1 -d"<"`
SIZE=`grep "groundspeak:container" "$TMPFILE" | cut -f2 -d">" | cut -f1 -d"<"`
DIFF=`grep "groundspeak:difficulty" "$TMPFILE" | cut -f2 -d">" | cut -f1 -d"<"`
TERR=`grep "groundspeak:terrain" "$TMPFILE" | cut -f2 -d">" | cut -f1 -d"<"`
COUNTRY=`grep "groundspeak:country" "$TMPFILE" | cut -f2 -d">" | cut -f1 -d"<"`
STATE=`grep "<groundspeak:state>.*<\/groundspeak:state>" "$TMPFILE" | perl -ne 'm/>([^<>]+?)<\// && print$1."\n"'`
# Now remove XML from the GC file
DATE=`grep "groundspeak:date" " $GCFILE" | cut -f2 -d">" | cut -f1 -d"<" | cut -f1 -dT`
TIME=`grep "groundspeak:date" " $GCFILE" | cut -f2 -d">" | cut -f1 -d"<" | cut -f2 -dT | cut -f1 -dZ`
sed -i '/groundspeak:log/d' " $GCFILE"
sed -i '/groundspeak:date/d' " $GCFILE"
sed -i '/groundspeak:type/d' " $GCFILE"
sed -i '/groundspeak:finder/d' " $GCFILE"
sed -i 's/<groundspeak:text encoded="False">//g' " $GCFILE"
sed -i 's/<groundspeak:text encoded="True">//g' " $GCFILE"
sed -i 's/<\/groundspeak:text>//g' " $GCFILE"
# Insert variables into the new GC file
echo "      		...Converting File"
sed -i "1i\Listing Name: $URLNAME" " $GCFILE"
sed -i "2i\GCCODE: $GCCODE" " $GCFILE"
sed -i "3i\Found on $DATE at $TIME" " $GCFILE"
sed -i "4i\Placed by: $OWNER" " $GCFILE"
sed -i "5i\Size: $SIZE (Difficulty: $DIFF / Terrain: $TERR)" " $GCFILE"
sed -i "6i\Location: $STATE, $COUNTRY" " $GCFILE"
sed -i "7i\Geocaching.com:$LINK" " $GCFILE"
sed -i "8i\ " " $GCFILE"
mv " $GCFILE" "$PUBLISH"
touch new
COUNTER=$((COUNTER-1))
fi
fi
done
echo ""
echo "			Reached 30 post limit!"
echo ""
# Pubish the new GC logs to Blogger
if [ -f new ]; then
echo ""
echo -n "Do you want to publish to Blogger (y/n)? "
read ANSWER
if [ $ANSWER = "y" ]; then
echo ""
echo "	Publishing to Blogger..."
echo ""
egrep -H "Found on [12][0-9][0-9][0-9]-" "$PUBLISH"/* | sort -k 3 | cut -f1 -d: | while read CODE; do
CACHE=`grep "Listing Name: " "$CODE" | cut -f2 -d:`
GC=`grep "GCCODE: " "$CODE" | cut -f2 -d:`
sed -i '/Listing Name: /d' "$CODE"
sed -i '/GCCODE: /d' "$CODE"
#google blogger post --blog "$BLOG" --title "$GC: $CACHE" --user "$USER" --tags "$TAGS" "$CODE"
echo "blogger post --blog $BLOG --title $GC: $CACHE --user $USER --tags $TAGS $CODE"
mv "$CODE" "$EXPORT"
echo "		Published: $CODE"
echo "$GC" >> "$EXCLUDES"
done
echo ""
echo "                  New logs published!"
else
echo ""
echo "                  Not published!"
fi
echo ""
else
echo "			No new logs."
fi
echo ""
rm -f *.out
rm -f *.tmp
rm -f "$EXPORT"/*.tmp
rm -f new

Curious Problem with Cork City Locale Geocaching Pocket Query

If you’re a premium member with Groundspeak’s geocaching.com then you will be aware of the ability to create pocket queries. These are really useful for throwing up to 1000 waypoints onto your GPSr for a days, or even a holidays caching.

As some of you know we were in Ireland recently (the reason for lack of posts) and I decided to take advantage of Stena Ferries free satellite wifi on the boat. I set a PQ for Cork and waited for it to build. When the query was ready I saved as usual to my shared Dropbox folder and then attached my Garmin 60CSx but despite loading fine onto my CacheBerry – the PQ just wouldn’y upload to the Garmin.

Puzzled, I tried an older PQ and verified that my script was working, as well as the Garmin was not having some kind of software fault. Next most obvious thing that occurred to me was perhaps the file was corrupted during download, after all it was a wireless connection via satellite on a moving ferry. I tested this theory by downloading the PQ again. Then I downloaded one I’d created earlier. The earlier one (Watford) worked, the Cork didn’t.

Then I decided to test the website PQ builder itself – maybe there was a technical problem. I checked the forums but there was nothing reported, so I posted a thread which didn’t go anywhere. I then tried creating an entirely new PQ of Leicester (never run before). I downloaded and Leicester loaded up onto my GPS no problem.

So there was something local to Cork which was borking my PQ which had to be in the file itself. I didn’t have much time left so I tried a splicing the file with my newly created Leicester PQ. I opened up the GPX file and chopped out the header:

<?xml version="1.0" encoding="utf-8"?>
<gpx xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" version="1.0" creator="Groundspeak Pocket Query" xsi:schemaLocation="http://www.topografix.com/GPX/1/0 http://www.topografix.com/GPX/1/0/gpx.xsd http://www.groundspeak.com/cache/1/0 http://www.groundspeak.com/cache/1/0/cache.xsd" xmlns="http://www.topografix.com/GPX/1/0">
<name>Cork</name>
<desc>Geocache file generated by Groundspeak</desc>
<author>Groundspeak</author>
<email>contact@groundspeak.com</email>
<time>2010-06-16T03:12:46.2642819Z</time>
<keywords>cache, geocache, groundspeak</keywords>
<bounds minlat="51.551967" minlon="-9.37105" maxlat="52.428967" maxlon="-7.593717" />
<?xml version="1.0" encoding="utf-8"?><gpx xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" version="1.0" creator="Groundspeak Pocket Query" xsi:schemaLocation="http://www.topografix.com/GPX/1/0 http://www.topografix.com/GPX/1/0/gpx.xsd http://www.groundspeak.com/cache/1/0 http://www.groundspeak.com/cache/1/0/cache.xsd" xmlns="http://www.topografix.com/GPX/1/0">  <name>Cork</name>  <desc>Geocache file generated by Groundspeak</desc>  <author>Groundspeak</author>  <email>contact@groundspeak.com</email>  <time>2010-06-16T03:12:46.2642819Z</time>  <keywords>cache, geocache, groundspeak</keywords>  <bounds minlat="51.551967" minlon="-9.37105" maxlat="52.428967" maxlon="-7.593717" />

and footer:

</gpx>

I then copied the remaining xml and pasted it below the header in the Leicester PQ, then saved this and zipped it back up.

I ran my script and and uploaded it to my 60CSx – it worked!

I can’t see anything wrong with the header info above but this is where the error seems to lie. Maybe it’s something to do with the coordinates in the boundary tag, or perhaps some hidden character I missed. Does anyone more technical in both xml and WGS84 Datum than me have an idea?

Oddly enough, subsequent PQs for Cork after we arrived worked too.

Just GPSBabel

A while ago I posted my frustrations with running GSAK on Linux, and how I’d found a way of getting the function to send to my Garmin 60CSx to work.

I thought I owed an update to anyone who read that to say I gave up on GSAK and now just use GPSBabel from the command line. Here’s the script I run:

#!/bin/bash

GPXZIP=$1
NAME=`echo $GPXZIP | cut -f1 -d.`

unzip $GPXZIP
GPXFILE=`ls *.gpx`
for FILE in `echo $GPXFILE`; do
    gpsbabel -i gpx -f $FILE -o garmin -F usb:0
done

I called the script ‘sendtogarmin.sh’. I saved it in the directory where I save all my pocket queries are stored as zip files. When it comes to load them, I just hook up the Garmin and then run:

sudo ./sendtogarmin.sh pquery.zip

Works for me, but I never bothered with using GSAK for much more than sending the pocket queries – I’m not at the stage yet where I feel I have to create complicated queries to get a good caching experience.

Welcome to wafitz.net

Well that was quick. My main blog has now been moved over to this domain, soon I’ll be picking up the other blogs and attempting to do some interesting things on this website. I’m working out a way to publish my previous Geocaching logs on a blog, which for now will be utilised through the xml-based pocket queries. But hopefully this can be a base for listing puzzles and multi caches too.