Out of the Blue - Michael Trafton's Blog

Last week, wecompleted a four-part Webinar series highlighting some of our recent Information Access projects with organizations like Dell, the University of Leeds, and the federal government. If you’re wondering what information access is all about, we’ve got recordings of these webinars online.

One of the webinars features yours truly giving a demonstration of FishFinder, the internal search application we use here at Blue Fish to discover and access important project-related documentation. FishFinder uses the Endeca platform to index all the documents in our Documentum content management system, and because it’s blindingly fast, gives our consultants near instant access to best practices, sample code, and other information that has been used on previous projects. This makes our teams more effective since they are not reinventing the wheel, and it allows us to complete our projects more quickly and with less risk.

If you’d like to know more about any of the solutions featured in these webinars, drop me a line (mikey at bluefishgroup.com).

I may have to eat a little bit of crow here. Recently, I made a point that Google Search Appliance (GSA) wasn’t very innovative, and therefore wasn’t very “Googely”. I still stand by the core of my opinion, which has to do with the way that Google presents search results. Because it can’t leverage the PageRank algorithm that they use on google.com, the search appliance is left to use the same relevance algorithms that other search engines use - noting the frequency and placement of the search term within the document and using that to determine how relevant the document is for the current search.

But there is more innovation in the Google Search Appliance than I thought there was. Dan Burton, a consultant here at Blue Fish, recently wrote an overview of the Google Search Appliance, and I learned a few things I didn’t previously know. One innovative technique they use is called KeyMatch, and it allows certain users to promote or highlight key content for a given search term. Similar to the way that Google Ads work on google.com, KeyMatch lets a user with the right permissions highlight a document, moving it to the top of the search results. If used correctly, KeyMatch can help companies satisfy their user’s most frequent searches by manually placing the most relevant result where they will easily see it. That’s pretty Googley.

The team has been busy lately, cooking up a tasty new batch of articles. These articles cover a range of topics from nuts and bolts programming with Documentum lifecycles and search APIs to guidance on how to deploy enterprise technology across multiple business units. There are also a couple of articles about search and information access - one is on Google’s Enterprise Search Appliance and the other is about simplifying the deployment of Endeca’s information access platform.

• DFC Search in Depth - Aaron has written a great article that explains how to use the DFC search APIs to query Documentum and process the results.
• Google Enterprise Search Appliance Overview - This article covers the features of Google’s search appliance and how it compares to other solutions on the market.
• Custom Java Lifecycle Modules - Explains how to insert custom business logic into Documentum lifecycle state transitions.
• Initiating an IM Program Across Multiple Business Units - Bryant explains how he’s been successful in the past when he’s had to deploy enterprise software across multiple business units or divisions.
• Simplifying Endeca Deployments with the Deployment Template - Until recently, deploying Endeca has been a bit of a black art. Learn how to make Endeca deployments easier with deployment templates.

The Blue Fish offices will be closed for the next few days. The entire company (more than 60 people when you include spouses and guests) is flying to Cabo San Lucas to celebrate a strong 2007 and enjoy some well-deserved rest and relaxation.

If you’re a top-notch consultant, software developer, or project manager who wants to work for a company that truly appreciates you, check out the positions we have open on our Careers page.

Last week, I gave a presentation at the Wachovia Services Summit, a conference full of Wachovia Bank branch managers that had come together to learn how other companies differentiate themselves through customer service. I was invited to speak about some of the client-focused practices we use here at Blue Fish. I was speaking alongside representatives from Nordstrom, Whole Foods, and Four Seasons Hotels, all of whom are famous for their great customer service, and I was honored to be included amongst such heavy hitters.

My presentation talked about the things Blue Fish has done to build what I call a Client Service Culture — an environment where we put our clients first and try to go out of our way to make them happy. It’s always fun to tell our story to people who have never heard it before. We’ve been doing some of these things for so long, it doesn’t seem very unusual, but I was surprised at the feedback I received telling me how innovative we are being.

For example, the Wachovia bankers were shocked to learn that the typical candidate at Blue Fish is interviewed by eight different people (for some roles, it can be as many as eleven). We do this not only to get a variety of perspectives on the candidate, but also so that our candidates can meet as many of their potential coworkers as possible. Culture fit is a two-way street, and we want to make sure that our candidates meet enough people to get a good feel for our company’s personality. Of course, we don’t conduct eight separate interviews. We have a few people in what we call a panel interview, and we send a couple more to lunch with the candidate. But by the time all is said and done, our candidates have gotten to know us pretty well.

I think Wachovia is being pretty innovative as well. How many companies do you know of that have a conference for their managers that is focused exclusively on customer service? You can tell that they care about their customers and are committed to continuous improvement. Wachovia is one of the fastest growing banks in Texas, and it’s largely due to their focus on the customer.

The Austin Business Journal has a nice article about Blue Fish in their most recent issue. The photo of me is a little silly (it was the photographer’s idea, I promise), but overall I think the article captured the spirit of Blue Fish pretty well.

The reporter, a woman named Laura Hipp, interviewed me about how I feel about the plans Google and PayPal have for hiring a bunch of software developers here in Austin. I’m a little worried, of course, but I’m not freaking out about it. I’ve always felt that hiring was about getting a great fit between the employee and the company, and I’ll bet that the people who are the best fit for us might be a fish out of water at those other companies (no pun intended).

When you look closely, there are several differences between Blue Fish and typical software companies. First of all, our teams are fairly small. A typical project at Blue Fish is made up of 3 to 6 developers. Contrast that to PayPal where there are over 300 developers working on their core payment processing engine. And there is a lot of variety at Blue Fish - our projects typically last less than six months, so developers are more likely to work with several technologies over the course of a year. Of course, we also have some beefy, multi-year projects for the type of developer that likes to dig in and focus on one problem for a long period of time.

Another big difference is that because we build custom software solutions, our developers are more in touch with the users of the software. Most software developers get their marching orders from a product manager, but at Blue Fish, our developers get to meet their users in person and truly understand their needs. I think this is one of the things that really attracts people to Blue Fish, and it has more to do with a developer’s personality than it does his or her skills. Some developers want to focus purely on the technical aspects of a problem, but those that do best at Blue Fish want to understand the underlying business problem and use that knowledge to design a creative solution. Someone told me recently that Blue Fish was the first place he had worked where the software he wrote was actually used - his time at his previous jobs had been spent working on products that never made it to market.

Another way that Blue Fish is different from Google and PayPal is our size - with less than 50 employees, we are tiny in comparison. Developers work across the hall from our executives, and I think they are more “plugged-in” to how the company operates than employees at other firms. For example, we share our company financials with all employees once a month. Once a quarter, our managers and executives have a half-day, off-site meeting the technical project leads to answer concerns and work on improving issues that are important to them (we do the same thing with our project managers). We are able to incorporate the ideas of our employees into our strategies and corporate initiatives in a way that larger companies just can’t do.

So Google and PayPal can bring it on, as far as I’m concerned. Competition is good - it will push us even harder to keep Blue Fish a great place to work.

In our Information Access practice, we often find ourselves implementing Endeca to help our clients find documents and other information easily and intuitively. Some of our engagements involve massive amounts of data, and it can be challenging to architect a reliable, scalable architecture when you are dealing with millions of records and gigabytes of data.

Dan Burton, one of our Solution Architects, just published an article full of tips on how to deploy Endeca in environments that have huge datasets. Learn more by reading Architecting Endeca for Large Data Deployments.

Over the years at Blue Fish, we have helped lots of clients migrate documents and other data from file systems, databases, and legacy systems into the content management systems we help them design, develop, and deploy. We call these efforts “Content Migrations”, and we have an entire Content Migrations practice area devoted to helping our clients perform these migrations quickly and effectively.

“Why does Blue Fish need an entire practice focused on Content Migrations?” you may ask. After all, the concept seems pretty straightforward. But these migrations can be a lot harder than than they look. I can’t tell you how many times we’ve had clients budget a week or two for their migration only for it to remain incomplete months later.

Pete Nevin, one of the consultants in our Content Migrations practice, just published an article detailing some of the common pitfalls we see companies make when they try to tackle a migration. The article is called Technical Challenges Faced During Content Migrations, and it discusses the typical approach most people use when performing a content migration along with several things to watch out for in each step of the process.

As you might imagine, here at Blue Fish, we “eat our own dogfood” by authoring and maintaining the Blue Fish Web site using EMC Documentum Web Publisher and Blue Fish Navigation Manager. This allows us to test the latest patches and releases in a real-world environment before they reach our clients’ production systems.

I’m sure you have also figured out that this blog is powered by some off-the-shelf blog software - in this case, it’s powered by WordPress, one of the most popular blogging platforms out there.

What you may not have known is that all the Enterprise Content Management articles that we publish here on the Blue Fish Web site are also powered by WordPress. We do this so that our readers can easily give us feedback by rating the articles and commenting on them.

Since we use Web Publisher to write our articles and WordPress to publish them, we needed to integrate the two platforms.

Marc Perlman just published an article on how we did this. It called Integrating Web Publisher and WordPress, and it’s a interesting read.

I love Google. I love almost everything about them. I love their motto (Do No Evil), I love their employee-focused work environment, and of course I love their web/image/news/video search tools. I also love most of their other products - Google Maps was ground breaking when it was introduced, Gmail has taken web-based email to the next level, and I use Google Reader everyday as my news and blog aggregator. I’m a devoted Google-lover because everything they do is so Googley.

But one of Google’s products stands out as very Un-Googley. You may not know it, but Google has packaged up their search technology into a standalone appliance that it sells for companies to use inside their firewall as an Enterprise Search Engine. Enterprise Search Engines are used to find corporate information that has been stored in various nooks and crannies of file systems, internal web sites, and content management repositories. Although Google has the best Internet search engine in the world, it has one of the worst Enterprise Search Engines. To find out why, let’s start from the beginning.

Back in the 90s, there were a lot of researchers thinking about search technology. Information on the Internet was growing at an astounding rate (it still is) and was driving the market for search tools through the roof. Back then, the main problem researchers were trying to solve was the “query expression” problem - that is, they wanted to make it easier for users to tell the computer what they were looking for. A lot of effort was put into natural language processing, and out of this research came search engines like Ask Jeeves, where the user could literally type in a question like “What store has the least expensive shoes?”

When Google came along, it blew these search engines out of the water. It turns out that all these researchers were trying to solve the wrong problem. Google showed us that the hard problem in search technology was not in expressing the query, it was in expressing the results - specifically, in placing the most relevant results at the top of the list.

Prior to Google, search engines determined relevancy by how many times the search term appeared in the web page and by where it appeared. If the word “Penguin” appeared in the title of the page and also appeared several times in the body of the page, that page was deemed to be more relevant to penguins than a page that only contained the word once. This led unscrupulous web masters to game the system, filling their pages with the most popular search terms repeated over and over, often in a tiny font the same color as the background. It wasn’t uncommon for these unscrupulous web sites to be near the top of a search engine’s result list.

Google uses a different approach for determining if a web page is relevant. Each time a web page links to another web page, Google treats that link as a “vote”. Web pages that get the most votes are considered to be the most relevant. This simple algorithm, called PageRank, uses a simple premise - if your page is the most popular (a lot of other pages link to it), it is likely to be the most relevant.

This approach works amazingly well for Internet searches, and I almost never have to look at the second page of results when I search for something on Google.

The problem with Google’s Search Appliance is that although the PageRank algorithm works great for web pages, it doesn’t work at all for Word documents, Excel spreadsheets, PowerPoint presentations and the like, because these documents don’t link to each other the way web pages do. And the information that companies most want to search for is contained within office documents like these. Google’s inside-the-firewall search appliances aren’t any better than Internet search engines of the early 1990s.

The biggest issue with all of this is that Google’s search appliances diminish their brand. When I use Google inside my firewall, I have to scroll through pages of results to find the document I’m looking for. This just doesn’t measure up to the expectations that I have for a Google product - it’s just not Googley enough.