Disclaimer

Every effort has been made to assure that all statements in this blog are true and factual accounts of interactions with various companies and individuals. The authors can not guarantee the accuracy or validity of any statements on this blog beyond that they are true to the extent of their investigation, knowledge and/or memories. The authors do not represent nor claim any connection to the aforementioned companies or individuals. Comments and poll results on this website are the sole responsibility of their writer and as under Section 230 of the Communications Decency Act, the authors make no claims nor do they accept any responsibility as to their accuracy. It is not the authors' intent to defame or otherwise harm any company or individual mentioned in this website, only to relay a non-defamatory chronicle of reported events.

Wednesday, June 11, 2014

Microsoft Bing Maps: Did they walk off with this startup's streetscapes?

In 2005, long before Google announced "Street View" or Microsoft announced "Streetslide", it occurred to the founder of Superhighstreet.com that it would be a wonderful use of web technology to be able to 'scroll down the street', namely to give any person anywhere in the world, a way to visit famous streets, and walk along them – no crowds, no rain, no hassle, no parking, no airfares. More so, he though it would be even better to see a 'head on' view of the shopfronts sliding past you (rather than the later Street View 'bubble' effect) for efficient browsing of building facades, and even to be able to click on each store's door, to open up their web store and shop 'inside'.

He left a well-paid job, and spent many months on this project, photographing famous streets like London's Oxford Street, coding the JavaScript, tying it all to Google Maps, and so on. In 2005 with great pride he finally launched Superhighstreet.com:

Superhighstreet.com in 2005 with its then unique horizontal streetscape view

(The name originates from the UK terminology of "the high street shops", meaning "the town center stores" and similar to the US "main street". "High Street" is the no. 1 street name in the UK.)

As you can see from the video below (also uploaded to YouTube in 2007), it featured a horizontal scrolling streetscape that was in fact much easier to understand than Google's later Street View, and a map with a 'you are here' man, which dynamically moved according to the GPS co-ordinates of the section of street currently in the horizontal display window. It also featured a 'Cross Road' button to turn to see the other side of the street.

Note: The jerkiness is down to the screen capture software used in 2007. The real site scrolls smoothly. This video was made as a promotional tool featuring celebrities on the street – a fun 'afterthought' but now valuable proof of copyright date.

If a user scrolls within the streetscape window quickly to the right, they can see how the squares of images load and unload spontaneously from Superhighstreet's server on demand. (This would later form part of a patent application by Microsoft.)

Superhighstreet were advised by a UK patent attorney that their technology was likely not patentable – advice they believe was given in error to this day, although it certainly would have cost tens of thousands to try to get all the claims accepted by the UK Patent Office.

They added a search page that was a nice rival to Google.com and Microsoft's then Windows Live Search, and even entered into partnerships with the stores featured on the streets, including Schuh (a big UK shoe retailer):

Superhighstreet spread far and wide – the founder was even interviewed on the BBC

On 1 August 2006, and after Superhighstreet went live, they contacted the Group Manager of Microsoft's "Seadragon" project, Blaise Agüera y Arcas, who at the time was developing a neat photography (not mapping) product called "Photosynth". Superhighstreet thought there might be some crossover and some room for them to exchange technology and do business together as both their products involved stitched imagery. Superhighstreet was used to doing business with high level firms via a previous company, which had done deals with the Virgin Group, and the UK's top 'high street' retailers for a previous invention.

A different one of Microsoft's Group Managers replied the very next day:


He explained that Superhighstreet had some things in common with something called "Streetside" that Microsoft's sister company, Virtual Earth, was working on. He provided a link to the "Live Labs" for that similar product, which looked like this at the time (2006):

Microsoft Streetside in 2006 with its non-horizontal street view

Certainly there are some things in common with Superhighstreet. There's a map under the image. The 'car' could be equated to Superhighstreet's 'man'. There's a photo view above the map of the real street.

Superhighstreet's map man
But one thing they had not hit on, evidently, in this live preview, was how beneficial it is to see just a horizontal streetscape, with interactive storefronts. In a way Microsoft's preview had more in common with what would become Google Street View, than it did with Superhighstreet. In Microsoft's version, one sits in the 'bubble' of a car, just like Google's Street View bubbles.

Content that Microsoft were going in a different direction (literally a head on direction, rather than horizontal), Superhighstreet continued to put effort and expense into adding streets to their site over the years.

Four years after they had showed Microsoft 'Superhighstreet', on 2 June 2010, MS applied for the first of 3 patents for something they were to call "Street Slide". Do you notice anything similar?

Microsoft Street Slide in 2010 with its 'brand new' horizontal method

This patent was ultimately to be granted on 28 Jan 2014. In Superhighstreet's opinion some of the claims in the patent are – perhaps coincidentally – the same as functionality that they revealed to Microsoft back in 2006:

Are the similarities between Superhighstreet (left) and the Street Slide patent (right) coincidental?

Although they certainly showed it to the key person at Microsoft, the MS patent does not reference Superhighstreet's material as prior art nor list the founder as one of the inventors, and they do not believe the examiner was aware of Superhighstreet.com. If he had been, in Superhighstreet's opinion the patent as it stands should not have been granted. Perhaps understandably, Superhighstreet do not have the $30,000 - $50,000 required to apply for a re-examination of the Microsoft patent.

This video shows Microsoft's first public announcement of their "Street Slide" project just after they filed for their patent, in July 2010:


Therein they lament the many same problems with 'bubbles' like Google Street View that Superhighstreet had foreseen before Street View was known about, and indeed problems that the 2006 preview Microsoft sent 4 years earlier also had. 

Then they present 'their' solution to these at the 0:37 second mark. To quote their video:
"...A single panorama ... does not allow you to get an overview of a complete street. Zooming in to look further down the street exhibits severe foreshortening and loss of resolution. Jumping from bubble to bubble [like Microsoft's 2006 preview] moves you down the street but still does not provide context. 
We present our system which we dub 'Street Slide' ... enabling efficient browsing of building facades ... Street Slide was found to be significantly more efficient than Google's Street View ... We use this space [above and below the streetscape] to add informational and navigational aides. The turn sign provides affordances for turning to see the other side of the street. Finally, storefront logos or graphics can be displayed below the strip. In the lower-right corner you see a mini-map. The center circle corresponds to the current location. 
Street Slide provides both an informative and immersive experience for exploring streetside imagery..."
The 4 main similarities highlighted:
Horizontal scrolling streetscape, map, 'cross road' button, store logo banners

These problems and the solution felt very similar to the problems the founder of Superhighstreet had imagined and then solved in 2005 with his own method for browsing building facades. In their opinion Microsoft's project had changed for some reason since their 2006 incarnation was sent. But obviously their product was significantly more advanced than someone with Superhighstreet's limited resources could code, compared to a company like Microsoft and their entire mapping team.

It was noted at this time that the contact at Microsoft that they had first reached out to with Superhighstreet, Blaise Agüera y Arcas, had been 'brought over' from the Photosynth team, to become the Architect of the Microsoft Bing Maps team. They wondered what had prompted his promotion to the maps team around the same time their map streetscapes became horizontal. Here he is presenting the very impressive other features of Bing Maps like telepresence in a TED talk, and there's no doubt he's a talented guy with an amazing product that resulted in a standing ovation at the end of the speech. We'd probably have stood up too.

In May 2011 a newer version was announced by Microsoft, renamed "Streetside" again, along with a video by Blaise Agüera y Arcas.

"Why the change?" Microsoft explain it's just better this way - the way Superhighstreet showed them in 2006.
http://blogs.bing.com/maps/?p=1375

2012: MS applied for two more patents for their version. These applications are still ongoing and have not been granted yet. To give just a few examples of many we have noted, which may just be coincidence or 'great minds thinking alike', Microsoft's patents carry "claims" which Superhighstreet feel they had shown MS in 2006, such as:

They feel Superhighstreet was 'combining the street level images to form the [horizontal] street view image' in 2005
In their opinion Superhighstreet had a 'metadata panel' (map that moves according to the GPS of the street view) 'below a bottom portion of the street view image' in 2005, as did it 'send the street view image to a client device', and so on
In their opinion Superhighstreet had a 'navigation user interface object to navigate the street view image in response to control directives' in 2005
The Superhighstreet 'navigation user interface' still live here

2013: Street Slide/Streetside goes live at: www.bing.com/maps


Streetside seems to struggle on Google Chrome

The rest, as they say, is history, and so is Superhighstreet.com who ceased further development in 2010 when Street Slide was announced, because they simply could not compete with the infrastructure Microsoft has behind them. Is that fair?

What do you think? Here's the timeline recap:
  • 2005: Superhighstreet launches unique horizontal streetscapes
  • 2006: They reach out to Blaise Agüera y Arcas at MS Photosynth
  • 2006: A colleague replies about non-horizontal MS "Streetside"
  • 2007-9: Blaise Agüera y Arcas moves from Photosynth to Streetside
  • 2010: MS patents horizontal streetscape, announces radically new horizontal Streetside
  • 2012: MS applies for 2 more patents. None reference Superhighstreet.
1. Microsoft Streetside before Superhighstreet

2. Superhighstreet

3. Microsoft Streetside after Superhighstreet


Microsoft's right to reply:
"We have investigated your claims internally and believe that they are inaccurate and legally unfounded." – Michael Brick, Senior Attorney, Microsoft

What do you think? In your opinion, did Microsoft steal this startup's idea?