Google Street View Tour of "Haven"

The SyFy TV show "Haven" was filmed using a variety of real-world locations around what is known as Nova Scotia's "South Shore."  A lot of the street scenes and store-front shots were done in the towns of Chester and Lunenburg, but there were many other locations used as well.  Below are links to Google Street View scenes showing a number of these "real-world" locations which stood in for the fictional town of Haven.

For some background on why I put this together, info on Google Street View, and general random thoughts please check out this post.

Changes since this was first published: Time marches on.  It's hard to wrap my head around it, but Haven ended filming around the end of 2014.  Google has updated many of the Street Views from October 2016 for Lunenberg and Chester.  As expected the Haven Herald signage is gone (you have to use the widget to look at historical views to see it). The 2016 shot of the Pew Arena shows a nearly empty parking lot, no RVs or anything parked there.  Nicki's Inn in Chester is up for sale.  The building next to the Chester Playhouse is painted green now.

Update for May 2018: In addition to the Street View scenes, I have added links to some user-submitted 360 degree panoramas for a few locations.

Update for December 2018: There are new (August 2018) views of the Grey Gull.  Note, I am intentionally leaving my link below to the 2014 version as it still shows the "Grey Gull" signage, but once there you can use the date widget to look at the new images.  As we knew, the signage is removed, the house looks empty and it seems like it will perpetually be up for sale.  Also the shots of the Cape Rouge in Bridgewater are now from August 2018.

Update for June 2022: Google Street View has added a little side road that runs right in front of the house used as The Rev's house in Season 1, Episode 2.  I am replacing the link and part of the description for "Hubbard's House Old Post Road" below.  The Grey Gull house has a chain link fence around it - based on the Street View image date it has been there since last year some time.  Apparently the new owners were having trouble with trespassers.  It's a shame - I've still never made it to Nova Scotia but I had long hoped to visit one day, and part of my plan would be to drive out to the Grey Gull and try to snap a few photos.

Lunenburg

Lunenburg is home to a couple of landmarks which are important to Haven, and has provided street scenes for numerous episodes.  The Lunenburg waterfront is featured prominently and frequently throughout the show as "the" Haven waterfront.  Although waterfront and aerial shots of other towns are sometimes used, Lunenburg is used for more of these than the others - especially as the backdrop for scenes of Audrey standing out on a dock.  The building used as the Haven Police Station is in Lunenburg as well as a church seen in the opening credits.

Lunenburg - Town Hall Blocks

There are city blocks in front of and behind the Lunenburg Town Hall which are pretty interesting and contain a number of locations from Haven, some of which are seen repeatedly:

• Lunenburg Town Hall Gazebo - This gazebo is seen in various episodes. 
• Lunenburg Town Hall (Haven Police Station) - Lunenburg's Town Hall served as the exterior for shots of the Haven Police Station.  The interior of the police station was a set located elsewhere.  This building is right next to the gazebo.  In between them is a monument to Lunenburg residents who served during the first world war.  The monument can sometimes be seen on the show but never well enough to read.  Note, in real life the arch over the door is very bright (almost white) and reads Lunenburg Town Hall, while the "Haven Police Station" has a red brick arch and a grey marble slab.  The Haven PD also has flagpoles out front.  At various times, shots of the building with the obvious Lunenburg signage and missing the flagpoles ended up in the show, usually aerial shots of the building.
• Lunenburg Courthouse (Haven Courthouse) - this is the north entrance to the same building as the Town Hall.  Seen in the episode where Duke is in court and attacked by lady justice, possibly others. 
• Lunenburg Town Hall Park Panorama - User submitted 360 panorama taken right next to the Town Hall in the park near the Gazebo and memorials.
• Lunenburg Park East End Panorama - User submitted panorama at the east end of the park.
• Homework assignment - I'm not providing a link for this, but try to navigate on your own - Use Street View to travel east on Townsend St. a few "clicks".  You should see the back side of the gazebo and a park area with grass and some benches.  This area was featured in the episode with the fireman whose trouble caused him to make other people spontaneously combust.
• St. John's Anglican Church - a couple of blocks west of the Town Hall, this church shows up in a variety of episodes (it's kind of hard to NOT show it sometimes), easily spotted in aerial shots of the town, but most famously it is the "burning church" in the opening credits.  The church actually did burn some years ago.  It was very hard to find a suitable Google Street View shot of this due to the surrounding foliage, but there are LOTS of nice photos of this church available.
• St. John's Panorama - User submitted panorama in front of the church.  There's another nice one inside the church...
• Intersection of Cornwallis and Lincoln - While there are far too many "random" street scenes from places in the area to include them all, I wanted to put this one in because it is featured numerous times during season 1 episode 1 (i.e. the pilot episode).  The view my link brings up is facing north- the scene where Vince and  Dave first introduce themselves to Audrey happens on the right-hand corner while on the left is the antique shop featured in the episode.  Also if you go back and watch the episode you will see a shot of the Bronco driving down Lincoln.
• King St. Mural - Head east down Lincoln St. until you come to King St. and turn north (back towards the Town Hall) and you will see this mural on the side of a building - it shows up in one or more episodes.  Look north on King St. to the intersection with Cumberland, you will see a TD Canada Trust bank with some green signage on it.  That building was used as the Haven Post Office in at least one episode.

Other Lunenburg Locations

• Waterfront (Bluenose Drive) - The Lunenburg waterfront is such a key piece of Haven, yet hard to display with Street View.  That's because we usually see it from the water side.  The particular spot I have linked is right where there is a pier used in some scenes.
• Lunenburg Angled Dock - Duke's dock - this is the dock where Duke's boat (the Cape Rouge) is located during the show.  Several things to note - the boat was not kept here year round, it was only here during filming.  In real life it sank (I think after season 4) and they only used old footage of the exteriors after that - the interiors were a set in Chester.  The linked Street View Image is from August 2012 with a ship that is probably Duke's - there are a number of later shots but they are missing the ship.  NEW LINK: Better View Of Duke's Boat from August 2012 (while filming for season 3 was taking place).  Not sure how I didn't think of this one sooner - it's from just down the street looking back at the dock.
• Duke's Dock Panorama - If I'm not sorely mistaken this user submitted 360 shot was taken on the dock where the Cape Rouge was moored during the show.
• Lunenburg Dock - they actually got a Street View camera out at the end of one of the docks looking back toward town, and I think this very dock was used in some scenes.
• Tannery Road Park - The Street View image may not look like much but zoom in and you will see this little spit of muddy land with the remains of a dock on it - this is where Audrey goes when she needs some alone time, and I think where the Colorado Kid was found.  I was able to verify this is the right location by looking at the buildings you can see behind Audrey and some of the other characters - if you spin the street view around you will see a little house with a red garage or mini-barn next to it.  The red barn is clearly seen in several episodes of the show.  The little house that is currently there is NOT what was there when the show was first filming - it was apparently torn down and rebuilt.
• Out Past Audrey's Dock Panorama - this is a user submitted shot taken out past the old dock, when the water is low.
• Lunenburg Academy - This very cool looking building is seen pretty clearly in the background of one episode, and is one of the most easily recognizable landmarks in the frequent aerial shots of Lunenburg used for Haven.
• Lunenburg Battery Point Lighthouse - NOTE: there are no publicly accessible roads leading to the lighthouse, so there are no Street View scenes.  There are a lot of photos of this lighthouse online, so this link is now a Google Image Search.  There are two lighthouses used in the show - the other being at Peggy's Cove.  This one at Battery Point is the one you see in the main credits, and it was used as the "Heart of Haven", where there was a cave underneath with a magical door leading to the Void...Of course it is destroyed at the end of Season 4, but somehow still shows up in aerial shots of "Haven".
• Intersection of Cornwallis and Fox - I picked a location for the view just a little south of the intersection.  This is where Jordan and the Guard confronted Nathan, Duke and Jennifer as they returned to Haven in Season 4, Episode 1.  The steeple of the church (Zion Evangelical Lutheran Church) is ripped off by a tornado in the sequence.  This is a lengthy scene in the show offering lots of viewpoints and vantages that you can compare with the Google Street View scene. 
• Kaulback Block - The "Mermaid Building" - during the opening credits there's a brief shot of the camera panning along a second-story row of windows with almost ghostly images of mermaids reflected in them.  If you have Netflix or some other way to pause it will be easier to confirm.  This is the building.

Chester

As with Lunenburg, various places in Chester were used for Haven scenes, and there are a number of points of interest grouped together around a couple of specific areas.  Chester is home to the Haven Herald offices, a couple of blocks on Pleasant St. were used tons of times in the show, and the local hockey and curling arena was home to a number of indoor sets used throughout the series.

Chester - Queen St.

• Mecklenburgh Inn - The Mecklenburgh Inn was Audrey's home base early in the show (before she started living above the Grey Gull) and was also used in numerous street scenes.  In season 1 episode 2 when the giant brass ball is rolling down the road, it is shown going by the Mecklenburgh Inn.  Just south of here is Chester's post office which occasionally shows up on screen as well.
• Queen St. Cluster - Ok, I just didn't know what to call this spot, but it's got a lot going on here.  When you click the link to bring up the Street View you should be looking at the building which served as "The Haven Herald" newspaper office dead-on.  If you go up the street north or south a click and look back you can see the sign hanging from the eaves, and if you zoom in you can still see the logo painted on the glass.  The link currently should bring up a Street View version from 2014 when the show was still being filmed - the signage has since been removed, and there is a 2016 version you can bring up that shows it is gone.  Spin the view 180 degrees - the house on the opposite side of the street shows up in a LOT of episodes due to the proximity to the Herald.  Look at the yellow building just south of and next to the Herald (Street View shows it with a sign that says "ABLE Engineering Services") - that building was used as Haven Joe's Bakery.  But in some episodes you see it with the ABLE signage...Go up the street north of the Herald and look back and you will see it is actually a "L" shaped building with a little parking lot - the entrance to the building buried back there in the corner of the "L" was the exterior of the "Gun & Rose" restaurant where Jordan was a waitress.  Finally - note this building has a red roof.  The L-shape and the red roof make this building really stand out in aerial shots, once you know it's there, you can't miss it!
• South Queen St. Monument - Go south on Queen St. - WAY south until the road dead-ends at the waterfront and turns into South St. - and you will see a neat little black monument.  Almost this exact shot is used in season 1 episode 2 when the giant ball is rolling along.  

Chester - Pleasant St. 

• Pleasant St. Cluster - another spot where there is just too much going on!  The view brought up in this link is pointing at a space between two buildings which are frequently seen on the show.  The one on the left - the Chester Playhouse - will be very familiar to the fans.  In the Christmas episode where the town gets stuck in a snow globe this building serves as the Haven movie theater.  During season 5, Mara stabs a woman in the eye with a pencil on the sidewalk right in front of the building.  Spin the Street View 180 degrees and behind you is the Kiwi cafe - which is seen in a number of episodes, but in particular was Big Benjy's Ice Cream in Season 1, Episode 4.   You see all these buildings in the episode with the people in bear suits walking around. 
• Nicki's Inn - Just west along Pleasant St. is Nicki's Inn, which appeared early in the series as "The Rust Bucket", which was partly destroyed by the giant rolling ball.  It also appeared as the "Black House Coffee" shop in Season 4, Episode 2 (where Jennifer bought coffee and found a number of burned bodies).  If you have trouble visualizing it as the coffee shop, they used the entrance in the alley to the right of the front of the inn.  Fans of the series who have followed Twitter and Instagram posts and interviews with the cast and crew may recognize this as the establishment run by Nicki Butler.  She was apparently loved by many involved with the show as well as being a local icon, and passed away unexpectedly back in 2015.  The establishment is now closed and the building was for sale as of late 2016.  Right across the street is a building that appeared as a restaurant during the show (The Irreverent Oyster).  Go up Pleasant St about a block further west and look back towards Nicki's - this is the scene in the final episode of season 1 when the ground opens up and swallows Max Hansen.

Other Chester Locations

• Chester Yacht Club Gazebo - that's probably not the right name, but it's the gazebo NEXT TO the Yacht Club.  It is used in a LOT of episodes.  It was the location of the Farmer's Market in Season 1 Episode 4.
• Chester Yacht Club Gazebo Panorama - User submitted 360 view inside the gazebo.
• Homework assignment - just west of the gazebo is a funny looking swimming pool (The Lido Pool) - do you recognize it?  To see it you may have to go south down Peninsula Rd. a half-dozen "clicks" and look back towards the northwest.  
• Church Park / Pew Arena - This is not a terribly interesting picture, yet in it you are seeing a place where the cast and crew spent a LOT of time - inside that building were housed a number of sets used in the series, including the interior of the Haven Police Station, interior of Duke's Boat, and interior of Audrey's apartment over the Grey Gull.  If you switch to the overhead Google Map view and use the "earth" view which shows the satellite photo you will notice this is a complex of buildings and there are tennis courts - these courts are featured in at least one episode, Season 1 Episode 5 where a woman's trouble causes men to age and die rapidly - a man dies right on the tennis courts there.  Just north and a little west is a large expanse with a couple of schools.  The Arena was also recently used for sets for the Lifetime movie "Sea Change".  
• MAJOR Homework Assignment - if you go west along Pig Loop Road looking at the huge space outside the arena, as you come to the corner, you can see a green Pontiac van shoved in the corner.  It's a little hard to see through the foliage and chain link fence, but it looks kind of beat up.  I'm 95% certain this is the van seen flipping over and tumbling down the road just over 21 minutes into season 5 episode 1.  Hopefully Google will keep this one archived - it's from August 2014.  Now, use the little widget to change to the version from August 2012.  If you look through the fence you will see a "Haven" ambulance.  And if you keep the date set to August 2012 and work your way back east towards the entrance, and keep looking towards that corner, behind the ambulance you will see...THE BRONCO.  BAM!
• Chester Middle School - It was hard to find a good Street View shot for this, as it's a bit back on the property where the Google car probably couldn't go, but this is the "blue school building" seen in a number of episodes.
• Chester St. Augustine's Parish - When the "Darkside Seekers" come to Haven, they set up their night shot at the beginning of the episode in front of this church.  Spin the camera 180 degrees and look at the yellow house on the opposite corner - they entered this house and encountered the "Rougarou."  A number of shots were done in that episode taken from various angles up and down the street by the church.
• East Chester Community Centre - Used as the "Dixie Boy Truck Stop" in multiple episodes. 

Hubbards

• Hubbards Shore Club - The Shore Club was featured in the episode when Nathan and Duke time traveled to the 1950's.  I believe the interior of the Shore Club was actually used on the show.
• Hubbards St. Luke's Anglican Church - This is The Reverend Driscoll's church on Haven.  It is featured in a lot of shows.  The giant rolling ball started out as a statue on the lawn here.  One of the things I love about Haven is how they tied together all these locations spread around multiple towns but made them feel like a single town. 
• Hubbards Bishop's Park Gazebo - Gazebo number 3!  Used in several episodes, just down the street from the Rev's church.  The scene in Season 4, Episode 2 where Duke tells Jennifer the story about a kid who broke his arm sledding when they were younger (the story is about Nathan) takes place here.
• Hubbards House, Old Post Road - This house was used as The Reverend Driscoll's home.  I am 90% certain they actually used the interior of this house.  The house is up on a hill which, when looking out through the front windows, would have a view of the Rev's church (St. Luke's).  Watch the episode (season 1 episode 2 I think) and you will see what I'm talking about.
• Hubbards St. Mark's Anglican Church - used as the outskirts of Haven in more than one episode.  Now spin the camera 180 degrees - in the current Street View image at the time I am writing this, which is shown with an image date of August 2014, there are HAVEN POLICE CARS parked across from the church!  COOL!  The blue Chevy Tahoe there is Dwight's vehicle on the show as well.  I *think* these cars were parked here for filming scenes for season 5, episode 14 (first episode of season 5b or season 6 or whatever we're calling it).
• Hubbards House at Tilley's Cove - The Grey Gull.  There are several decent shots of the building from the road, just walk the camera up and down.  Also, use the little widget (in the upper left on my browser) that lets you view historical images from previous years.  The "current" images right now are from August 2014.  However there are a number of really nice shots from April 2012 when the Grey Gull sign was up, and there was less foliage covering the building.  There are also some shots from 2009 prior to the start of filming the show, when the building doesn't have the sign - or windows or doors apparently.  In photos I've seen elsewhere since these Street View images were done, the signage has been removed.  June 2022 Note: There are new images of the house which has been painted, has perhaps new siding and roof, and is now surrounded by a black chain link fence and clearly marked "Private Property".  I had wondered how long people would be able to continue wandering down to the house to take pictures...
• The Grey Gull Panorama - User submitted panorama taken at the porch corner by the water, ground floor.

Other Areas Of Interest

• Peggy's Cove Lighthouse - Another lighthouse used in the series, in Season 1 Episode 5, where a woman's trouble causes men to age rapidly and die.
• Mahone Bay Centre - This building appears in numerous episodes as the Haven Medical Center.  Note that early in the show the building was doctored up with signage that said Haven Medical Center, a flagpole with a US flag, etc.  But similar to the issue with the Lunenburg town hall, later aerial shots of this building are missing the Haven-related signage.  
• Mahone Bay Street Scene - This is one of the many street scenes from all over the South Shore area to be featured in Haven - in particular this one is seen in the episode where Audrey relives the same day repeatedly and various people are getting run over ("Audrey Parker's Day Off")...Be sure to zoom the view out all the way and pan around, and you may recognize more scenes.
• Robinson's Corner Farm - this is the home of "The Barn", the supernatural construct that plays such a big role in the show but which is seen only a few times considering.  It's located pretty far back on the property off the road, and presented a challenge to find a good Street View angle.  In Season 2 Episode 3 when the "real" Audrey Parker finds the barn, it is shown onscreen flipped left-to-right, but definitely still the same building.
• Halifax Dingle Tower - Dingle Tower in Halifax was used as the Armory in the final episodes of Haven.
• Halifax Boondocks Diner - The Boondocks Diner is where Nathan and the "copy" of Audrey (created by Croatoan to trick Nathan) were eating at the opening of Season 5B Episode 13.  The diner in the show was also called Boondocks, but they hung a "Cleaves Mills" sign out front - nice Stephen King reference...
• Bridgeport Cape Rouge - The Cape Rouge belonged to Duke Crocker, serving as both his home and an important part of his business as a smuggler.  During the winter months when the show was not being filmed, the boat was docked here in Bridgeport (during filming it was docked at the "angled dock" in Lunenburg, liste above).  Note - the boat partially sank in March of 2014 but it was apparently raised, since the linked Street View image was captured in September 2014.  If you have any doubt this is the Cape Rouge (that was actually the original name of the boat), check out this article about the sinking.  There's a series of 10 photos - number 7 is a good shot for comparison the the vessel pictured in Street View.  This site featuring the work of Katherine Bickford has a few photos of several ships lined up along the wharf in Bridgeport; the third ship (in the back of the line) is the Ryan Atlantic II, otherwise known as the Cape Rouge.
• Corner of Montague & Prince - OK, if there is a "Most Obscure Location" on this page, I think this is it.  Take a look at this building.  If you don't recognize it, take a look at these pictures: http://www.farfarawaysite.com/section/haven/gallery1/gallery1/gallery4/gallery.htm, and then come back to the Street View scene.  Spin the camera around.  Those promo shots for the first season were done right in that building in the linked Street View scene.  It used to drive me crazy wondering where they were done.  I was absolutely convinced they were done in Chester, but turns out it was right there in Lunenburg. This building was used for some early promo shots of the three main actors.  Unfortunately the site I used to point to for those shots is gone.

Conclusion...

And that's all I have for now folks.  If you know of a location that was significant to the show and that you'd like me to add, please let me know.  I'd like to give a special thanks to Pauline P. whose Haven Photo Project  got me to thinking I should share these links, and who kindly helped with a few locations that had eluded me.  Also I would like to to thank Michelle Diesbourg for some locations as well as additional detail about how some were used.

Notes on the Google Street View Tour of "Haven"

A while back my wife introduced me to a TV show called “Haven” which aired on the SyFy channel.  Haven is about a fictional town in Maine where a portion of the residents suffer from supernatural curses called “the Troubles”.  Through the experiences of FBI agent Audrey Parker, we learn about the town and its people, what the Troubles are, where they come from, and how Audrey is connected to the town.

Instead of being filmed on a studio back lot somewhere, Haven’s outdoor scenes - streets, storefronts, houses & yards, etc. were filmed in the real world, specifically in and around several communities that are part of Nova Scotia’s South Shore.  Chester and Lunenburg provided most of the principal and recurring locations, but there were also places around Hubbards, Mahone Bay, Robinson’s Corner, and more in between.  These locations not only made Haven seem more real, but they have a charm that is captivating in its own right, and (no doubt like many other people) I wanted to learn more about the area, and perhaps to visit some day.

Since these are real places, (the exteriors of) most of them have been cataloged by Google’s Street View.  Street view is an adjunct function of Google Maps and Google Earth, which allows you to zoom down to a specific map location, bring up a 360 degree panoramic view, pan around (and up and down), and “walk” the view up and down the street or road.  Google has a fleet of vehicles (mostly cars, but in some cases bicycles or other) that drive around, with special cameras and GPS trackers, imaging as much of the world as can be seen from the road.

I have found a number of places that were used in Haven - some of which were used repeatedly and frequently in street scenes, aerial views and the like, and a few that were used less often but still significant.  I collected links to these scenes, which when clicked should open your browser and go straight to a particular Street View (including the direction and zoom level I had set when I made the link).  

To get the most fun out of this you will need to familiarize yourself a bit with Google Maps and Street View - and the easiest way to do that is to just play around with it a bit.  Click one of the links.  Pan around (click and hold left mouse button on main picture and drag around and up and down), zoom in and out (scroll the mouse wheel while pointing at main picture), “walk” up and down the road (when the mouse cursor is “on the road” look for the circle with an arrow and click to move).  Look for the “film strip” below the main image and click on Street View links provided by other people or images they’ve posted.  Above all, have fun - this is all virtual, just explore and enjoy!

Some semi-random thoughts:

• IMPORTANT - do not travel to the South Shore of Nova Scotia hoping to meet the cast or crew, or to see an episode of Haven being filmed.  The show has ended, filming ended around the end of 2014, and the final episodes aired in 2015.  If you want to go and walk some of the sidewalks your favorite characters walked, or if you just fell in love with the area and want to experience it for yourself, by all means make the trip.  But nothing you see on this site (or any other) should be construed to mean that there is still any active filming of Haven occurring - there’s not. In one or two places the Street View images linked from my blog may still show Haven-related signage, but in real life and in the present day, it's all gone.
• Likewise, if you travel to Nova Scotia do not be disappointed if the real world locations don’t look exactly like the Google Street View photos.   Just like any photographs, these represent a point in time - a specific time on a specific day in a specific year.  (Street View photos show you the “image date” when they were taken so you can see how old they are.)  Over time, Google "re-images" sites, and they (usually) keep (some of) the archived, older versions of how locations looked available. When I find a site and create the HTML link for it, it records not only the location and such, it should point to a SPECIFIC version (date/time) of the scene. So if the version available on Google Street View when I made the link was from August 2014, that's what you're (hopefully) going to see when you click the link - even if Google has since created new images.
• The older versions of locations can usually be accessed with a little on-screen "widget" that sits in the upper left (it shows the month and year the current image was created). Click on it to access other views - older ones and maybe newer ones too. This may allow you to see some shots taken during the time the show was being filmed, so you may see "Haven" signage on some buildings, the paint jobs on buildings (and cracks in the street and so on) will match what you see on the show, etc. Also because the Street View cars are in motion when the images are taken, the views are rarely in exactly the same spot - so sometimes you get "better" views in different years.
• The streets themselves are pretty much public, and there are various parks and gazebos that you can access.  However, some locations are private property - just because they’re visible from the road doesn’t open them to tourist traffic.  For example, the "Grey Gull" house is privately owned, “the Barn” is on private property, and so forth. For some of those you may only be able to take pictures from the road.  If you travel to the South Shore, enjoy the publicly available locations, but don’t trespass.
• When finished, the show will have aired 78 episodes.  The show featured many locations around the NS southern coast.  An exhaustive list of locations is beyond my ability to provide, and would probably be too much anyway.  That being said, just by learning the layout of a few places, you’ll soon be recognizing scenery in a LOT of shows.  These locations got used over and over, maybe just from slightly different angles.  It starts to feel like a familiar place.
• Or perhaps I should say familiar places - knowing that some scenes are shot in Lunenburg and some in Chester, and more important WHICH are shot in which places, can kind of throw off the continuity.  This is especially true when you see an aerial shot of Lunenburg followed by a storefront on a street in Chester, or a shot of the school in Chester followed by a street scene in Mahone Bay.  Speaking of aerial shots, you will perhaps have already noticed that Lunenburg and Chester look quite different from the air, and yet both are depicted as “Haven” (I think maybe Mahone Bay aerial shots are used as well).  Whether this will affect your ability to enjoy re-watching the show is something I can’t control or predict - but if you are worried about it, this kind of Google “pixel peeping” is probably not for you.
• At the risk of stating the obvious, Google Street View cameras are VERY different from television film or video cameras, and the filming is done under entirely different conditions.  The Street View cameras tend to use fairly wide-angles, while a lot of the TV shots are done from a distance and at a very high zoom.  TV shots are done when the lighting is “just right” as well (or lighting may be manipulated artificially), whereas Google Street Views are mostly on relatively clear days and when things are pretty evenly lit.  In other words, what you see on Google Street View may only bear a passing resemblance to what you see on the show, and you may have to use a little imagination to figure out how things fit.

Three Things

I still don't know what this blog is about, other than being a place to work out some stuff rattling around in my head.  So far I have avoided writing about politics and social issues, but the last couple of weeks have been a wild ride for us here in the U.S.A.  I grew up, live, and work in "the South", where the majority of my family, friends and co-workers are unabashedly conservative, religious, and mostly Republican with a smattering of Libertarians rounding out the crowd.  They are also, for the most part, very good people with whom I can have thoughtful conversations on a wide variety of issues, and the fact that I often fall into the liberal camp doesn't seem to have cost me any friends.

Still, I've really been thinking a lot about some of the things that have transpired in this great nation of ours of late, and I'm going to use some space here to clear my head.

Supreme Court Ruling On The ACA

The affordable care act, which many refer to as "Obamacare", has been a huge political football.  It was (I believe) a well-intentioned law, but nevertheless one which was pretty badly flawed.  I think it will be a long time before we can assess the overall level of benefit it provides or the damage it does.  And I suspect we will continue to see court challenges to parts of the law which some folks find objectionable, so the final form has yet to take shape. 

Recently the Supreme Court of the United States (SCOTUS) heard a challenge to a provision that allows the federal government to provide subsidies to people who buy insurance on the exchanges that have been set up.  A poorly worded provision of the law literally refers to exchanges set up by the states, which led to a suit claiming that this means the federal government isn't allowed to give subsidies to people who buy insurance on exchanges set up by the federal government, which is what we get in states that refuse to set up their own exchanges. 

This week the court ruled that the federal government can give those subsidies.  Many on the right have howled about this - that in making this ruling, SCOTUS has "rewritten" the law.  Frankly, I think this is a misunderstanding about how SCOTUS works.  In cases like these, their goal is generally to try to understand what the authors of a law meant or intended.  The court decided, rightly in my view, that there is no way the authors of the law intended to withhold subsidies from people forced to buy insurance on federal exchanges just because their states didn't set up their own.

Flying the Confederate Flag

The Confederate Flag is a symbol which means different things to different people.  It's a gross understatement to point out that the mixture of emotions it evokes is complex; for people like me, liberal southerners, it's really complex.  For some the flag is a symbol of a time when our country's laws allowed one race of people to own people of another race, when the southern states tried to leave the Union specifically to preserve that right, and of more than a century since that war in which racism has remained alive, like Voldemort, half-dead but never quite killed off.  For others, it is a symbol of a time when their ancestors took up arms and risked life and limb to defend their homes from an invading army, and acquitted themselves honorably in battle.  And frankly, for some, it's simply a symbol of a mysterious attribute we might call "southern pride."

Over the last week in the wake of a terrible mass shooting in which a young white man brutally murdered a number of black churchgoers, there has been an intense discussion concerning the meaning of the flag (which is featured prominently in pictures of this young man), and concerning whether and when it is appropriate to display it.

I believe in free speech, and therefore believe that for individual citizens, there is no question but that they have the right to fly the flag in their yards, wear clothing using the flag, put bumper stickers on their cars and trucks, etc.  However when it comes to government institutions, be they state, county or municipal, I believe strongly that it is wrong for them to fly it.  There are two reasons for this.  First, the flag is also a symbol of rebellion against the Union, and any state or other government flying the flag is making a statement glorifying rebellion against the U.S.A.  The fact that a number of southern states have done and continue to do so is almost incomprehensible to me, especially given the extreme level of patriotic fervor felt by nearly all southerners.  Second, these governments are charged with the representation of ALL of their constituents, of all races and all ancestries, and to fly this flag is a direct insult to vast numbers of their citizens.  I have no desire to see the flying of the flag made illegal for individual citizens, frankly do not think it should be a legal matter for the states or municipalities either - just that those government institutions should make the clear and rational choice not to do so.

Supreme Court Ruling on Gay Marriage

This week the Supreme Court ruled that the individual states may not prohibit gay and lesbian marriages.  There is much left to happen here; there will be challenges and court battles.   But make no mistake, I have expected this.  I believe that it is inevitable that we will have marriage equality in the U.S.A.  This is where the currents are carrying us, and while some may swim against the river for a while, eventually it carries us all along.

The trend over more than a decade has been towards a growing percentage of the American public in favor of legalized gay marriage.  For the past year most polls have put this number at around 60%.  That number is interesting.  Some folks who are unhappy with the SCOTUS ruling this week feel that the court abridged the natural democratic process that should have been allowed to play out. What I think it interesting is that at least in the Senate, 60% is what we think of as a "supermajority".  Assuming the trend continues, Americans would almost certainly have elected an increasing number of Senators and Representatives who would have supported gay marriage, and congressional action would have occurred at some point.  It would have taken years longer of course, but this is what I meant by seeing this as a sort of inexorable current.

More to the immediate point, in my view there was very good reason for the court to make this ruling now, rather than allowing the country to go through the longer process.  Over the last few years, either by direct popular vote or due to legal action, well over half of the states were allowing gay marriage.  A situation in which a couple can marry in one state, and then find their marriage to be held illegal or invalid in another, results in a sort of chaos which is not good for anyone.  It's easy to say things like "Well, they should just choose to live in a state where their marriage is welcome."  But this ignores the fact that the choices we make about where to live are driven by forces not always under our control.  What happens to the gay couple when one partner needs to live in a particular state for a job?  What happens when they need to move closer to family, perhaps to care for an elderly parent?  The simple fact is that some kinds of legal arrangements need to be accepted by all states in order for the nation and her citizens to function.

Finally, let me just say this - my own feelings about homosexuality were resolved long ago when I realized one simple fact - it's not a choice.  Ethical and moral questions were simply made irrelevant by that one thing.  In this country we can and do make legal distinctions about the choices people make, but we do not make legal distinctions about who people are.  Saying that a person must "choose between who to love and where to live" makes no sense once we accept that people do not choose who to love.  Until now, the only choice for gays and lesbians has been whether to live a closeted existence, denying the reality of who they are in exchange for the convenience of benefits, or to live openly and be punished by a country with a confusing and constantly changing set of laws.

The SCOTUS ruling doesn't set everything right, not by a long shot - but it clearly sets forth the path we are going to follow.  For my gay and lesbian friends, I'm happy for you.  For the rest of us, and especially the people I know who are wondering whether the country is now on the verge of some sort of moral collapse - I say, "We're going to be just fine, like we always have."

Printing Problems Redux

Some time ago I wrote a rather lengthy post about an old case where I had to troubleshoot a difficult problem with print jobs failing - The Case of the Silence on the Wire.  Recently I have had to look at a problem that carried some of the same baggage.  An external customer is printing to a print server at our print facility, over a VPN, and seeing some issues.

This is a LPD/LPR setup, with our server listening on TCP port 515 and the customer's system using the standard client side ports 721-731 (see https://www.ietf.org/rfc/rfc1179.txt).  When the issue was reported a few weeks ago I didn't really see anything I could put my finger on.  Our server response time (as calculated by my analysis software) was a little slow, and there were a few retransmissions from the client, but overall the connections looked healthy enough - TCP three-way handshakes looked ok, data being transferred with our server acknowledging, proper FIN exchanges at the end.  At least that's what I saw the first two or three times we were contacted to check it out.

Today I was asked to take another look - the problem was reported to have occurred between 7:00AM and 9:00AM on June 23rd.  I grabbed a capture off our sniffers and took a look.  The sniffer is capturing everything both on the inside interface of our firewall and on the interface that goes to our private extranet connections, including VPNs, so I had a sort of "double trace", with one copy of the traffic showing the connection to the real internal IP address of our server, and the other showing the connection to the external NAT address.

Looking at a list of connections that took place during the time in question, I saw a bunch of connections that looked more or less like what I described above, but today I noticed something different - there was a connection listed that looked really small on the packet count.  This was sourced from client-side port 722.  I filtered on the trace and saw incoming SYN packets, but our server wasn't responding with SYN/ACKS - it was responding with plain ACK packets, and the ACK numbers weren't correct for the incoming SYNs.

Now, first guesses aren't always right and you must take care to check things thoroughly.  On the other hand, when you've been working with a particular system as long as I've been working with TCP communications, you can sometimes get pretty close to the mark.  In this case, I wondered whether the server was responding to an old connection - some previous connection sourced from port 722 that didn't close properly, so the server was still trying to reply using ACK numbers for that old connection.

I began working my way backwards through what our sniffer had captured - all the way back to about 6:35AM - and every time they tried connecting from port 722, we were just sending these ACKs.  In all cases the ACK numbers we were sending back were the same (at least on the packets taken from the inside interface of the firewall), and none of them were anywhere close to being correct for the SYN packets.  Earlier than that there were no connections on June 23rd.

I shifted focus to my Netflow tool.  Hunting for connections out of the vast number captured on a big capture box being fed by multiple taps can be really difficult, but Netflow boils down connection information to the essentials.  My Netflow records indicated that communications from this client had ceased a little after 10:00PM on the night of the 22nd, and further that there had been a connection from client port 722 that had enough packets in it to have been viable.  With that information I delved back into the sniffer to find that connection.

It was exactly what I was looking for.  The connection had started off fine, with a good three-way handshake, and for a time had proceeded normally - client sending data, server sending ACKs.  At the end, the client send a final packet with data in it and with the PUSH flag set indicating the server should process it and acknowledge, which it did.  I verified that this last ACK got through our firewall, as it appeared in the trace taken on the outside.  After that the client didn't send anything further for about 25 seconds, no further data and no FINs - after which, it sent a new SYN packet from client port 722.

This packet did not get through the firewall - the firewall was monitoring state and still thought the old connection was open (the firewall has an idle timer of 1 hour for TCP connections in the state table).  The firewall - apparently - sent an ACK using the last valid sequence numbering from the previous connection.  The client resent the SYN several times, and the firewall sent those ACKs each time, never letting the SYN through to our server.  After a while the client gave up and moved on to a new port number.

The difference between this and what we were seeing in the morning was this - by the time the client got around to starting up again on the 23rd, the firewall had forgotten about the old connection, removing it from the state table, so it was now allowing the SYN packets through to the server.  But the server still had the old connection open (more than 8 hours later!!!) and was still sending ACKs for the sequence numbering of the old connection from the night before.

There were two interesting things about these traces that revealed something I'd never seen before, and which challenge my assumptions about how much I know about this particular brand of firewall.  First, I've never seen the firewall send an ACK of its own to a connection like we saw on the final connection on the night of the 22nd.  At that time the firewall was not letting the SYNs through and my traces on the inside interface of the firewall confirm this - the print server was not getting them and was not sending the ACKs, yet I could see ACKs on the outside interface of the firewall.  As near as I can tell the firewall had to be sending them.

Second, in my experience this particular type of firewall is very strict about keeping track of state, and I would not have expected it to allow those ACKs from the server the next morning - once the firewall was letting the SYNs through it should have been watching for SYN/ACKs from the server, and also watching to make sure the ACK numbers were correct.  Instead, it was letting those ACKs go right through.  I am thinking maybe the firewall is programmed to do this in case there are out-of-order packets on the wire, but it still seems a little freaky and I'm going to have to read up on it.

Panasonic DMC-LX100 - First Impressions

Note: This is not a full review of the LX100 as I lack the technical expertise to do it justice.  For a couple of good, in-depth technical reviews, check out this one at DPReview and this one at Camera Labs.

I got into photography as a hobby as a young man, and over the years I've owned a number of cameras, including a very capable DSLR with some fine lenses.  One of the hardest things to accept has been that I just don't like carrying around a big camera and a lot of associated gear.  I love the freedom of full manual control, but I hate the weight and bulk of the traditional setup.  For these reasons, earlier this year I finally sold off my DSLR and lenses (which spent most of the time on the shelf gathering dust) and went looking for something that would better suit my needs.

My short list of desired features went something like this:

• Something small, light, portable
• A good quality non-removable lens 
• Full manual controls
• Good low-light sensitivity (I never learned to properly use a flash and thus prefer natural light)
• Minimum image size of 10 megapixels
• The ability to shoot RAW as well as JPEG

The camera which I chose, and which ticks off all the boxes for me, was the Panasonic DMC-LX100.  I have had the camera for about a week, and today I'm going to lay out my overall impression of this camera.  

Controls & Handling

The LX100 is small - not shirt-pocket small, but probably coat-pocket small.  However it is large enough to fit comfortably in my hands which are medium sized for an adult male.  There's a comfortable grip on the right-hand side and hand-holding feels quite normal, even though almost all my previous cameras have been a lot bigger.

The LX100 gives full manual control of the things a photographer cares about - ISO, aperture, shutter speed, and focus.  One of the neat things about this camera is that several of these controls are laid out in a traditional way - the shutter speed is selected with a dial on top of the camera, the aperture using a ring on the lens, likewise a ring for manual focus (this ring serves various functions depending upon the mode in which the camera is used), and an exposure compensation dial on top.  Shutter speed and aperture can be fine tuned using a dial on the back of the camera - in other words you can select values in between those found on the physical controls.

The camera allows for a full range of settings from full manual to full automatic.  Shutter priority, aperture priority, manual or automatic focus (with an array of autofocus modes), manual or automatic ISO selection, it's all there.

Shooting with the LX100 feels very familiar - after selecting some combination of manual and automatic settings, I aim the camera at my subject, use a half-shutter press to evaluate exposure and focus, recompose the shot if necessary and shoot.  It's a satisfying experience.

As with nearly all digital cameras, many features can only be accessed through a menu system, others through a variety of dedicated buttons on the camera.  Initially I found the menu system a bit daunting, but it's pretty well organized and I quickly got familiar with the locations of those features I most frequently use.  There is also a "quick menu" feature that allows you to customize an abbreviated set of features you're interested in.  

One thing I really appreciate is the electronic viewfinder.  While the camera's LCD screen is decent, my eyesight is not, and the ability to look through a clear, bright viewfinder with a dioptic adjustment is a big deal for me.

Performance

The LX100 is a serious piece of equipment, despite being labeled a "point and shoot" on Panasonic's web site.  It is outfitted with a really nice, fast Leica lens with a maximum aperture of f1.7, and that's paired up with a micro four-thirds sensor.  This camera does really well in low-light.  Image quality appears to me to be very good, as good or better than my old DSLR, although my own capabilities are not often up to the task of getting the best out of the camera.  In short I do not feel like the camera is holding me back.

There are certain things I have had to learn to pay attention to when using this camera.  The two biggest are autofocus and automatic ISO.  In both cases the camera does not always make the choices I would prefer.  

The camera has a 49-area autofocus which should - in theory - produce good results, but I have found myself using the single-area setting more and more, which allows me to point the camera at my subject and use the half-press shutter method to evaluate focus.  One thing I do appreciate is the ability to focus manually using the lens-mounted ring, and the camera has a neat "focus-peaking" feature than makes evaluating precise focus on macro shots much easier.

As for ISO, the camera has a tendency to err on the side of selecting higher ISO settings than I would like.  I have taken some shots of roses in my yard, in plenty of light, only to find later that the camera selected an ISO of 1600 and they turned out grainier than I wanted.  Setting the ISO is certainly easy enough, so when I know I have plenty of light I just pick something suitably low.  On the other hand, I find the image quality to be better at higher ISO settings than I was ever able to achieve with my previous cameras.  Indoor hand-held shots at ISO 1600 have produced some very acceptable results.

Although I highly prize all the manual controls, the camera does a reasonable job in full-auto mode, and it has a dedicated button for activating that.  Press it once and the camera becomes a real point and shoot, so snapping spontaneous shots (or handing it over to a non-photographer) is simple.  Pressing it again returns the camera to the previous state.

One of the outstanding features of the camera is that it has a nine-leaf aperture, which produces very round and soft out-of-focus areas (what real photographers call bokeh).  I love taking close-up shots of subjects with a large aperture setting and getting that lovely, soft look to the background.

One of the few things I would change if I could is I'd like a little longer focal-length.  The effective zoom range is 24mm to 75mm.  I'd like it be able to do optical zoom out to about 100mm.  You aren't going to get up close to subject with this camera unless you carry it up close - taking pictures of this birds and squirrels on the feeders in my front yard isn't so easy with the LX100. 

Interesting Features

Although I mainly bought this camera for its ability to operate like a traditional all manual camera, there are a number of things I've found to be pleasant, even fun surprises:

• Wireless control - The LX100 can be paired with an Android or IOS phone or tablet, offering some interesting possibilities.  When paired, you can view what the camera sees on the remote device, adjust some (but not nearly all) settings, and shoot.  If you are shooting something on a tripod and want to avoid shaking the camera by pressing the shutter button, this is the way to do that.  It also means if you want to be in a group shot, you can set up the camera and take your time getting situated and fire it off.  The wifi app also lets you transfer images and movies to the phone or tablet for easy upload to social media sites.  The pairing process, however, can be a bit of a pain.  I have it working pretty reliably now but only after a lot of trial and error getting the process down.  I might do a separate blog post on this at some point.
• Time-lapse movies - I am not writing about the camera's considerable movie-making features because I rarely take videos, but one of the neat things you can do is full in-camera time-lapse photography.  You set up the camera on a tripod or otherwise secure it, adjust the normal features (shutter speed, aperture, ISO, focus) or set some of these to automatic, pick how often you want shots to be taken and how many you want it to take, and let the LX100 go to work.  At the end of the process you are prompted to process the images into a video, with selections for the video quality and frame rate, and even whether to make the video in forward or reverse order of images.  One of the neat things is that you can go back after the fact and reprocess the images into videos with different settings.
• Panoramas - The camera lets you take panoramic shots, panning as it shoots, and it stitches the images together automatically.  So far I have found this produces good quality panoramas.  Shooting panoramas by manually stitching images together in an external computer program may produce better results, but I always found it too cumbersome to mess with.  But since the LX100 makes it so easy, I have found myself using it on occasion and I'm happy with the results.

Accessories

I have outfitted my LX100 with a few accessory options.  I put a B+W UV/Haze filter on the lens and it will likely stay there for the life of the camera.  I know there are a variety of opinions about this, but to me, it's cheap insurance for the lens.  When I sold off my DSLR equipment earlier this year, I was able to guarantee my buyers that the lenses were completely clean and free of scratches because I did this with every lens I bought.

I am not fond of traditional lens caps, which are easy to lose, so I was interested in getting a more-or-less permanently mounted auto-opening lens cap for this camera.  Panasonic makes one, but theirs is about $40 US, and according to various reports will not close over the lens with a filter installed (such as the UV lens I use).  I opted for an aftermarket cap, the JJC ALC-LX100.  It fits well and closes over the UV lens perfectly.

I bought a Gariz "Mirror Less" leather wrist strap for the camera.  Although the LX100 comes with a traditional neck strap, it is a small enough camera that it feels quite comfortable on the wrist, and the Gariz strap is a good choice.  It fits well and it looks very classy.

I bought two spare batteries (aftermarket replacements for the DMW-BLG10).  Battery life varies a LOT depending upon how you use the camera.  If you're just taking photos, especially without flash, it may last quite a while.  If you are shooting video, doing a lot with wifi, doing time-lapse shots and processing into video, all these things can knock the battery down really fast.

Finally I picked up a Lowepro Format 110 bag just to keep the camera, batteries & charger, and USB cable all in one place.  

Final Thoughts

I am really pleased with the Panasonic DMC-LX100.  It fills my requirements for portability, manual control and image quality.  It's got more digital tricks than I have time to write about here, and which will provide endless learning opportunities.  My list of complaints is minimal; most of the things other reviewers have complained about (lack of external mic for videos, lack of touch screen LCD) just don't matter to me.

More importantly, it's got me doing photography again.  I carry this camera with me, because I can.  It's easy to pick up any time I have a few spare moments.  The manual controls and great ergonomic layout make the camera a joy to hold and use.  I am back to doing photography, because the LX100 gives me the right set of tools to do it.  

Some Things Shouldn't Have To Be Hard

Part of my job involves supporting the network for a business unit with government contracts.  We have a connection to a private government extranet, over which our users connect to several websites required to fulfill the contract work.

Monday morning the senior director of IT operations for this business unit called me to say that his users couldn't log into one of these sites.  He had already been in touch with tech support for the site, and they had confirmed that it was up and running, and suggested we had a problem on our end.

I started my troubleshooting by logging into the perimeter router connecting to the private extranet, and saw that the connection was up.  Next I logged into a perimeter firewall and checked that there was live traffic passing in both directions - everything looked healthy there as well.

Finally I logged into a PC on the affected network and tried connecting to the external site myself using a web browser.  I was unable to connect.  Browsers these days do a pretty poor job of indicating what the problem is if a site can't be reached.  I was using IE 11, and it gave me a list of possible causes that covered just about every possible issue.

I decided to look up the IP address of the remote site so that I could trace the path through the network and double check firewall rules.  Using nslookup at the command prompt, I got a good indication of the problem right away - I was unable to resolve the IP address of the site.  My computer was configured to point to our internal DNS servers, which in turn forward certain domains to DNS servers located across the private extranet.  

Since I was unable to resolve the IP address, I suggested that we needed to get the on-call DNS administrator to check things out.  In the meantime we also started a conference call with the tech support people for the remote network.  While waiting for our own DNS administrator to join, I described the issue I was seeing.

The remote technician asked me, "Well, what did you change?"  I told him we hadn't made any changes.  He asked, "Did you do anything to your network connection?"  No, we hadn't.  "Did you make any firewall changes over the weekend?" was the next question.  No, we didn't.  I reiterated to the remote tech that our connection was up, everything seemed to be working, but we just couldn't get DNS resolution.

After a short while our local DNS admin joined the call.  In short order he confirmed that the DNS servers were working properly, no changes were made on our end, and we seemed to be getting "denied" messages back from the remote DNS server.  The remote tech repeated just about every possible iteration of the question about what WE had done to break things.

Only after more than an hour of this line of questioning did the remote technician finally reveal that the remote DNS servers had been changed over the weekend - completely replaced with entirely new devices.  It took a little longer, but it was eventually discovered that the new devices had a built-in ACL which was blocking our requests.  The old servers hadn't had this capability, and the ACL which the remote DNS admins had put in place didn't allow our servers to talk to theirs.

So riddle me this, Batman - you know you changed out your DNS servers, but when I call and tell you my DNS queries are being refused, you spend an hour making me repeatedly assert that I didn't change anything?  I lost two hours of my time, and more importantly my business lost two hours of productive work for dozens of users trying to fulfill their quota of work on a government contract because some bozo didn't want to admit that his change broke the system?  Priceless.

Solving A Common Problem With Chromebook Smart Lock

I have a Toshiba Chromebook 2, and when I saw the new Smart Lock feature I really wanted to try it out.  Basically Smart Lock allows the Chromebook to be automatically unlocked when it is within Bluetooth range of an Android phone.  Android phones themselves have a similar feature that can be activated by proximity to a Bluetooth device, within a particular geographic location, etc.  I was therefore a little bummed to find that I couldn't use Smart Lock because my phone, a Samsung Galxy Note 3 on Verizon, was still on Android version 4.x - Chromebook's Smart Lock only works with phones running Android 5.x (code named Lollipop).

A couple of weeks ago Verizon finally pushed out the Lollipop update for the Note 3, so naturally I immediately tried setting up Smart Lock...and it didn't work.  The procedure goes like this:

You need an Android phone running Lollipop (duh).  The phone and the Chromebook both have to be registered to the same Google account.  They have to be within Bluetooth range of one another.  And they both have to be connected to the Internet.

When you activate the feature (by going into the Chromebook's settings - advanced section and clicking the setup button for Smart Lock) the first thing that happens is the Chromebook locks itself, requiring you to supply your password.  Then a dialog box comes up with some notes about what you're doing and a button to "Find your phone."  And this is where the problem showed up - each time I clicked it, the setup tool reported that it could not find my phone.

Browsing around online I found people suggesting that you need to pair your Chromebook and phone prior to setting up Smart Lock (the Google instructions do not say this).  It didn't work.  Some folks suggested you have to turn on Smart Lock on the phone itself (also not according to Google).  That didn't help either.

I opened a support ticket with Google, and the first thing they suggested was that because I was running on the Chromebook beta channel, they couldn't help, and that I would have to switch to stable (necessitating a powerwash).  When that didn't help, they suggested a "hard reset" on the Chromebook (pressing the refresh and power keys at the same time) and a reboot of the phone - still no joy.  After that the support person didn't have anything new to offer.

As I continued searching for a solution I found a number of people who claimed to have had the problem and to have solved it by wiping their phones - doing a factory reset.  This seemed a little extreme but it began to suggest a solution.  Finally I found some information that it might have something to do with how the phone is registered with Google, specifically with "Google Play."

Basically, when you first set up an Android phone with Google, it reports itself to Google, and the information is used with Google Play services to show you only those apps which will work with your device.  But somehow the process of getting a phone upgraded to Android 5.x doesn't update the information they have stored about it.

The reason this matters is that when you click "Find your phone" during the Chromebook Smart Lock procedure, it isn't looking for your phone via Bluetooth - it is checking with Google Play to see if you have a phone registered running Lollipop.  Those folks wiping and reloading their phones were basically forcing the phones to re-register themselves with Google Play services.

It turns out there is an easier way to do this.  What you need to do is to force Google Play Services on your phone to update itself.  The way to do this is to essentially uninstall Google Play Services on the phone, and reinstall it.  On most phones, the way to do this is to go into the settings for the phone, then the Applications area where you can see all applications running on the phone.  Tapping on Google Play Services will bring you to a screen where you can uninstall it.  Note that on most phones you are not completely uninstalling Google Play Services, but rather uninstalling all the updates it has received.  You can also do a "force stop" on it.  After doing this, Google Play Services should update itself on the phone.  And when it does, it will register with Google Play the fact that you are running Lollipop.

Once this is done the Chromebook should be able to find the phone.  It worked for me, it has worked for others, and I hope it will work for you.

UPDATE: I forgot to mention that when I went to "uninstall" Google Play Services, at first I couldn't because the buttons were greyed out.  This was because I had tried to set up Smart Lock on the phone itself, and the phone was now using the services.  To turn it off, I had to go into Settings - Security - Trust Agents and disable Smart Lock on the phone there.  Once I did that I was able to proceed with uninstalling.

Attack Of The Clones (Microsoft Takes On Chromebooks)

Tech news and business websites are posting lots of stories these days suggesting that Microsoft is "targeting" Chromebooks with a new line of cheap laptops.  The HP Stream, priced at about $200, was the first major entry in the category, and now we're seeing reports that Microsoft is partnering with hardware vendors to create laptops well below that price.  The term "Chromebook killer" is cropping up all over.  Microsoft seems to be particularly worried about the adoption of Chromebooks in school systems, where the combination of utility and ease of administration makes them an obvious choice.  A new generation of computer users is getting regular exposure to Chromebooks, learning that they are capable devices through daily use, and those young people will grow up to be consumers one day.  It's easy to see why this is a real concern.

Here's why I think it's a losing game for Microsoft:

1. Chromebooks aren't about "cheap": It's true that many Chromebooks are inexpensive.  But their low price isn't necessarily the primary goal - rather, it's the logical result of the architecture.  The point of a Chromebook is to leverage cloud server-based computing and storage, more or less turning the laptop into a terminal device.  Chromebooks have cheap, low-power processors simply because they can.  And not all Chromebooks are "cheap" - my Toshiba Chromebook 2 with 1080p IPS display costs over $300, due largely to the spectacular display and better-than-average sound hardware.  In building cheap Windows laptops, Microsoft is missing the point - lots, maybe even most of the people buying Chromebooks aren't buying them because they're too poor to afford anything else.  They're buying Chromebooks because they like what they offer.
2. Chromebook buyers see the lack of Windows as an advantage: Windows is bloated.  It requires a lot of resources to run well.  It's popularity, and aspects of it's design, make it a favorite target for hackers.  It has to be patched constantly, and the patching process is cumbersome and fraught with problems.  It can't be run safely without add-on applications to guard against viruses and other malware.  It gets fouled up with use over time.  And in environments where IT staff have to administer large-scale deployments, it takes a lot of work to manage. Chromebooks aren't like that.  They are relatively stable.  The OS is replaced on a regular basis rather than patched, and the process is very smooth and quick (simply requiring a reboot that takes, on average, about 10 seconds).  Chromebooks are relatively safe from malware, and offer a simple way to remove all customizations and return to factory configurations (called the powerwash).
3. Windows needs resources: Microsoft likes to point out all the things you can do with Windows, but not with Chromebooks (warning - the link above contains several factual errors).  And it's true - you can't install full versions of Microsoft Office on a Chromebook.  You can't install Photoshop on a Chromebook.  You can't install commercial, power-hungry games on a Chromebook.   What these comparisons fail to point out is that while you can install these things on a cheap, sub-$200 Windows laptop, running them with acceptable performance is another matter entirely.  The last time Microsoft decided to try this, the market was flooded with cheap "netbooks" which many users found frustrating to use.  In some ways, these new, cheap Windows laptops will be even less capable than those netbooks - Microsoft is pushing the use of super-low-power CPUs like the Bay Trail Celerons (which do fine on tablets and Chromebooks, but which are not predicted to run intensive applications like Photoshop all that well).  The new laptops are coming with internal storage reminiscent of a Chromebook - like the 32gig SSDs in the HP Stream - and offering free cloud storage for 1 year, similar to Chromebooks - but that storage is paltry for the installation of applications like Photoshop and Microsoft Office.  They come with 2GB of RAM, barely enough to get Windows up and running, and in no way adequate to run Photoshop or advanced games.  The best way to summarize the problem is that in order to hit the desired price-point, Microsoft has had to lower the hardware resources to such a low level that the Windows experience is likely to be sub-par.  They may garner some sales in the short-term, but people expecting to run Office, Photoshop, or triple-A game titles will be disappointed.  My guess is this will hurt Microsoft in the long run.

It isn't my conclusion that Chromebooks are going to kill off Windows, and few Chromebook users think so.  However I do believe that Chromebooks fill an important niche, and I just don't see any way that Microsoft can play in that space without sacrificing Windows' strengths.  It will be interesting to see how this plays out.

Two Weeks In Belize With Chromebook

Early in March I traveled to Belize to visit my girlfriend and enjoy a break from work.  This was my first opportunity to travel with my Toshiba Chromebook 2, and to rely on it as my primary computing device for an extended period.  I carried the CB2 in my CaseLogic 14-inch Checkpoint-Friendly Laptop Bag. I'd already been using the CB2 and the bag for a while each day going to work, but relying on these things for a couple of weeks away from home would be interesting.

I'll address the performance of the bag first - simply put, it was perfect.  Although the bag - designed for a 14-inch laptop - was a lot smaller than the huge laptop backpack I used to carry, it was just right for my needs.  In it I had my Chromebook and charger, a mouse, my Kindle (Paperwhite verison 2, I like e-ink e-readers better than tablets for reading), my wallet, passport, and a bunch of little stuff - pens, paper, keys, loose change, etc.  Fully loaded, the bag was still lighter and easier to move around than the old backpack.  Access to my stuff was always super convenient.  The bag easily fits under airline seats.

The main feature of this bag is its "security friendly" design, which means you don't have to take your laptop out of the bag (if you use it correctly).  The laptop fits in a compartment in the rear of the bag all by itself, and when you are putting it through a security scanner, you unzip the bag such that the laptop compartment lies flat to one side by itself.  As long as you have put nothing else in the compartment, the security scanners can "see" the laptop without obstruction.  And when the bag exits the scanner you can grab the shoulder strap, pick it up, and it (more or less) "closes" so you can just walk away.  I went through airport security checkpoints 3 times with this bag and at no time did anyone take issue with it, in America or Belize.

Finishing up with my thoughts on the bag - with the one caveat that this particular bag is designed for a 14-inch laptop and thus is intentionally smaller than other bags - I highly recommend it.

The Chromebook was similarly a joy to use.  I won't go into what Chromebooks are all about here - if you like the idea of a laptop designed around cloud-based services you will understand.  The fun started when I got on the American Airlines flight from Miami to Belize and found that they had GoGo in-flight Internet.  My Chromebook came with 12 free passes for GoGo and so I got to try out the service.  It worked well - and was very weird, sitting there posting to G+, and doing Hangouts chats with friends at 30,000 feet.  Shortly after we cleared Cuba (about an hour into the flight) the service dropped out - the GoGo page seemed to indicate that they don't have rights to offer the service everywhere.  Those free passes are only good for one year from the purchase of the Chromebook so I don't know if I'll get another chance to use them, but it was really neat.  

Since the Chromebook is mainly for use online, Internet availability obviously is a concern - "offline" apps notwithstanding, nearly everything you would want to do with a Chromebook really does require an Internet connection.  My girlfriend has DSL and a wireless router at her home on Ambergris Caye, so most of the time connectivity was not an issue - we did spend some time getting the phone company to work on her connection as she was getting only getting about 1mbps despite being signed up for 2mbps service.  

Pretty much every morning, I would grab my coffee and head out to the porch to enjoy the beautiful weather, listen to the birds singing, and catch up with the world.  I would check my email (with Google Inbox), read the news on CNN, catch up with friends and family on Facebook, follow my Google+ groups, the usual stuff.  All of this is very normal, mundane activity - and it may be accurately pointed out that it could all be done on a smartphone or tablet - but the experience on all of these is better with a full-fledged browser like Chrome.  Having a "real" keyboard is a major plus, especially when posting.  And although I've never really liked trackpads, I got so comfortable using the Toshiba CB2 trackpad that I never bothered hooking up my mouse.

During the visit I tried streaming from Hulu and Netflix.  Hulu's geographic restrictions prevented me from using my account - I have yet to sign up for a good VPN service - but Netflix worked fine even with the 2mbps DSL connection.  I also had a number of movies stored on my SD card and the Chromebook playback was great - this model has a 1080p IPS screen and everything looked wonderful.

In short, the Toshiba Chromebook 2 performed perfectly as a portable computing platform.  For me the Chromebook has always been about having a useful secondary device, never meant to replace my desktop.  I have it on my desk at work every day, and occasionally use it in the living room at home.  However, for the two weeks I was in Belize, I found it to be great for everyday use and never once found myself wishing for anything more.

James's Rules of Troubleshooting

While digging through some old documents this afternoon, I came across something I had written about 5 or 6 years ago.  It's a pretty fair summarization of some things I have learned about troubleshooting.  I've made a few minor edits, but for the most part it's just as I wrote it back then.

James’s Rules of Troubleshooting

Before - (things to do/know prior to a problem – when something breaks it’s too late to start working on these):

• Know your stuff.  Be an expert on the technology for which you’re responsible.  “SME” means Subject Matter Expert – be one.  It’s MUCH easier to spot what’s wrong if you know what “right” looks like.
• Know where to find your product manuals, configurations, and logs.  Know what’s in the logs. Know how to read them.  Make sure logs are tuned to show the right amount of data, kept for a sufficient period of time to be useful, and are time-synchronized with everything else in the network.  Keep frequent backups of configurations / changes.
• Have the tools you need.  Have them installed.  Have them up-to-date.  Know how to use them. Try not to get tied to a single tool, no matter how good it is (when all you have is a hammer, everything looks like a nail).  Know more than one way to skin the cat.
• Work on your people skills.  You got into I.T. to avoid dealing with people?  Hopefully I’m not the first person explaining to you that this is not possible.  The network, computers, and software exist to serve people (commonly known as “users”) and you will need to be able to deal with them.  This includes people on other teams and from other technical disciplines. Network people need to be able to talk to server people, Unix people need to be able to talk to Windows people, etc.  Don’t let personal issues fester – they’ll get in the way at the worst times.
• Work on your communication skills.  Be able to speak and write clearly.  Clarity and accuracy are very important.  To the degree that it is possible, clarity is achieved by being as simple as you can be while retaining accuracy.  Get comfortable standing in front of small (5 to 10 people) and medium sized (10 to 25 people) groups and explaining how your technology works, how it ties in with the rest of the system, etc.  Be good at drawing diagrams.  Have a diagram of what your stuff looks like before you ever need it.

During (what to do while working a problem):

• Get a clear description of the problem.  This is often harder than it sounds.  “Users”  can talk in vague terms – “Everything is slow.” “The network is broken.”  You’ll need to elicit the right information through direct questions.  “Exactly what were you doing when the problem occurred?  Were there any error messages displayed on screen?  Were other applications affected?”  You may have to repeat questions multiple times in order to get the user to answer what you’re asking.
• Oddly enough, it can be even harder to get a clear story from a technician – they may be giving an edited version of events based on their own bias (they think their stuff can’t be broken, or that they know where the problem lies).  They will also likely want to tell you all the steps that were tried before you were called.  Unless they kept very good records of what was done, in what order, and the results of every test, that information is likely to be less than helpful.  If you don’t feel you can completely rely on the source, no matter how good your relationship, it is best to do your own investigation – “See for yourself!”
• If this is a problem with something that has been previously working and is now broken, find out what changed (if anything).  Software updated (on server or workstations or network infrastructure)?  Hardware changed?  Don’t be too quick to dismiss something that you don’t think is related – the installation of new DNS servers really can be the cause of slow network performance logging into Unix boxes.  Use your company’s change control record-keeping system to research.
• Check your stuff first.  When you are asked to join in a troubleshooting effort, make sure your components are not misconfigured or broken.  When someone asks you to take a look, your response should not be “My part can't be broken…” – rather, it should be “I’ll go check that out and get right back to you.”  If you are prepared (see the “Before” section) this should take very little effort.
• If the problem turns out to be your area, you need to fix it, but you also need to report honestly and accurately to the team or leadership.  I won’t tell you which one to do first – that depends on what’s broken, the rules at your company, etc.  But you should make that report a priority. Honesty can be difficult when the problem reveals a personal error.  All I can do is urge you to suck it up and do the right thing – it’s worked remarkably well for me over a long period of time.  People understand that if you’re not making the occasional mistake, you probably aren’t working.  They will respect honesty (as long as you’re learning from your mistakes and not making the same ones over and over).
• If the problem is not in your stuff, offer to help other areas however you can.  If you got called in because you have a reputation as a great troubleshooter, this will already be understood.  But even if that’s not the case, you may have valuable insight to offer – or maybe just a fresh set of eyes (and brains). If you can’t help directly, you can still learn a lot by watching the process and observing the resolution.
• Try to recreate the problem.  If you can’t test things in the production environment, try to set up a test-bed.
• Compare working and non-working configurations.  Got two servers that are supposed to be doing the same thing, but one isn’t working?  Find out how they are different!
• Be persistent.  Don’t give up.  Stick with the job.  
• Know when to get help.  Some I.T. people are loathe to open problem records with a vendor (including yours truly).  But if a production system is down and the bottom line is hurting, bring the vendor in earlier rather than later.  There maybe a known issue that they can recognize quickly.
• Take breaks.  During a protracted issue, you’ll want to rest your brain on occasion.  Failing to do so can cause you to overlook otherwise obvious problems – when you look at something for too long, it starts to look normal even if it’s broken!  Get up and walk.  Drink water.  Don’t forget to eat.

After – because eventually, the problem will be solved…

• FIX the problem.  If workarounds were applied, remove them.  Patch software.  Reconfigure equipment.  Whatever was broken, make it whole again.  “Bandaids” that are applied to get through an initial rough spot should not be considered a complete fix.
• Bring the system back to “standard”.  Don’t settle for a one-off solution that no one will remember exists in a week.  If there is something wrong with the standard, FIX THE STANDARD.
• Document what you did.  The standard documentation for the technology in question should be updated to reflect new configurations, new software versions, etc.  Diagrams should be updated.  The problem isn’t fixed until this is done.
• Be able to express in clear language to your superiors and peers what happened, the steps taken to fix it, etc.  Make sure you understand what happened, in technically accurate terms.
• Review your performance (and that of your peers) during the troubleshooting process.  Did you discover systems that aren’t time synchronized?  Fix it! Did you discover you really DON’T know how to use that nifty tool?  Practice!  Did you take too long to come to the right conclusion, perhaps overlooking data that was obvious in hind-sight?  Review the process you followed and try to understand how you might have gotten to the correct solution more quickly. The after-action "lessons learned" session is a valuable opportunity, not to be wasted.