Queues are for the Brits, Part 1

del.icio.us Tags: ,,

First off, I need to say that I sincerely hope that no one takes offense to the title of this post.  The title is meant to be a play on the phrase, “gone to the birds,” not an ethnocentric slur.  The alert reader will also pick up on the double entendre that in Britain, queue is a much more common word.

I have stated in earlier posts that I lead a team of top-notch developers.  My primary responsibility is early research and architecture; my team are the ones to transform the vision into reality.  We recently completed our pilot project, which I plan to blog about at some length.  One of the requirements that we met was the ability to design an advanced ACD that takes into account any number of variables and implements a “best-match” algorithm.  We believe this to be a significant advance over traditional algorithms of the skills-based routing.  In fact, a multiple-hour search yielded an awfully slim amount of relevant data.

The Problem with Skills-Based Routing

The primary problem with queues is implicit in the name: most queues operate as first-in-first-out mechanisms.  Before we can address why FIFO is a bad idea for contact centers, we need to have some context.

There is an even bigger problem that is a fundamental part of the queue problem.  This problem is in the existing call distributor algorithms.  A simple example will illustrate:

Assume we have three available agents.  Agent 1 is licensed to sell insurance in Michigan and Indiana.  Agent 2 is licensed to sell insurance in Indiana and Ohio.  Agent 3 is licensed to sell insurance in Indiana only.  Assume also that 80% of our inbound calls are from Michigan residents, 10% from Indiana residents, and 10% from Ohio residents.  If a call comes in from a resident of Indiana, any of the three may be able to sell an insurance policy to the caller.  However, only one is best suited in this context to sell the insurance policy to the caller: Agent 3.  There is a 90% chance that the next call will be from either Michigan or Ohio, meaning that Agent 3 would not be able to handle the call.  That means that we want to reserve those agents whose skills are most in demand for the calls that demand those skills.

Many contact and call center software providers trumpet their algorithms for call distribution, but most of the algorithms are some variant of, “How do I spread the load of calls evenly over my agent pool?”  The question that should be asked is, “How do I find the best possible agent to handle this call?”  Some software providers have started to deal with this question and have introduced a second factor into skills-based routing.  By assigning a rating to an agent’s skill, there is indeed additional intelligence available to the call distributor.  Consider the following example:

Assume we have three available agents, with skills rated as shown in the matrix.

Agent Windows skill level Office skill level Internet skill level
Agent 1 70 40 10
Agent 2 10 70 40
Agent 3 40 10 70

 

Upon receipt of an inbound call, the skill matrix is analyzed and the agent with the highest rating in the skill necessary is assigned to the call.  Therefore, if we receive a Windows call, the call is assigned to Agent 1.  If we receive another Windows call while Agent 1 is still occupied, Agent 3 is assigned.  If all agents become available again and an Internet call is received, the call is assigned to Agent 3.

In many senses, the second example is still vulnerable to the problem noted in the first example.  Assume our call load is 90% Windows, 5% Office and 5% Internet.  If a call comes in for Windows, it should be rightly assigned to Agent 1.  However, if a subsequent call comes in for Internet support, it might be more appropriate to assign Agent 2 to the Internet call and reserve Agent 3 for the next Windows call.

The fundamental problem with traditional skills-based routing is that it takes so few factors into account.  In part 2, we will consider a potential solution to this problem.

Interact 2008

INTERACT2008BlogArt2

This morning, I received an invitation to Microsoft’s Interact 2008 conference that is taking place from April 8-10 in San Diego, California.  Those of you who have been monitoring this blog for the past month know that I’ve been on the very-fast track to learning about Microsoft Unified Communications.  Although we primarily are developing against Speech Server and Office Communications Server at this point, we envision writing some code against Exchange Web Services to bring e-mails into our CRM also, truly unifying communications.  What gets me fired up is the fact that we truly have the ability to do whatever it is that we want to do.  I’ve said it before and I’ll say it again: what I like most about Microsoft is that they give you a solid foundation, a blank slate, and then they step back and  get out of the way.  Our biggest problem is convincing the business people in our company that even though the sky is the limit, we need to contain things at a more manageable level.

If you’re headed to Interact 2008 and you work in the contact center space or are interested in call distributors, I’d love to take some time to meet up with you and chat one-on-one.  If there is enough interest, we may even be able to form a small community of our own to address contact center concerns and OCS.  It seems that we could all benefit by coming together and building a solid community.  I will continue to blog about our experiences in developing a call distributor, blended predictive dialer, and other software, but we need to get more voices out there.  Feel free to ping me on my blog through comments or by e-mailing me at definitejunkmail (yes, definitejunkmail) on GMail.  I’ll reply back with my work e-mail so that we can stay in touch.

Looking forward to meeting some of you!

OCS Roles Primer, Part 1

One of the first things I really struggled with when ramping up on Microsoft Office Communications Server 2007 was understanding roles, their responsibilities, and how roles overlap.  What roles do I need?  How many of them do I need?  Can an edge role be installed on a mediation server?  What’s more, Microsoft tosses out diagrams like the following with very little explanation:

The information that is available out there about OCS server roles is scattered across many different documents, Web sites, and blog posts.  Hopefully, this post will distill information on the roles available in OCS, their coexistence constraints and hardware/software requirements.  That said, let’s get a little bit of vocabulary out of the way first.

  • Office Communications Server: A Microsoft product designed to facilitate communications both inside and outside the office.
  • Presence: A metric that takes into account both your availability (available, idle, away) and your willingness (available, busy, on a call) to communicate.
  • Endpoint: Any device (SIP phone) or software package that registers itself with Office Communications Server as belonging to a user, meaning that the user can be contacted through the device or software package.
  • Enterprise Voice: Probably the most noteworthy addition to the product since 2005; allows calls to enter and exit Office Communications Server.  This means that from any endpoint, users can make or receive calls to traditional phone numbers.
  • Public Switched Telephone Network (PSTN): The traditional telephone network that delivers telephone service over dedicated copper cables.

 

Pool Server Roles

In enterprise-class deployments, multiple servers are typically assigned the traditional front-end server roles.  Although this is sometimes referred to as a cluster, the term that has been assigned in this case is “pool”.  As an aside, I would recommend that any business who is seriously considering OCS for Enterprise Voice use an enterprise-class deployment even if scalability is not a concern.  Resiliency is also something you gain when you deploy multiple servers playing the same role.

Front-End Servers

The front-end server handles a lot of the plumbing for OCS.  It authenticates users against Active Directory, routes invitations to appropriate endpoints, and generally coordinates all other features of OCS.  It is important to note that once a front-end server has assisted in routing the invitation to appropriate endpoints, it gracefully steps back and allows the endpoints to communicate directly.  This means that a front-end server will not be overwhelmed by handling thousands of simultaneous video chats.  It merely directs the invitations and coordinates the set-up of communications, then allows the endpoints to communicate however they may choose.  (The exception to this rule is instant messaging sessions, which always travel through the front-end server for archiving purposes.)  It is also important to note that a front-end server may handle merely these responsibilities or more if the deployment team decides to consolidate multiple roles onto the front-end server.

Specifically, the front-end server:

  • Authenticates users against Active Directory
  • Aggregates and disseminates presence, contact, and other similar information
  • Routes invitations to appropriate endpoints (or gateways) and cancels other invitations once an endpoint has accepted
  • Tracks status of sessions (even though the session communicates from endpoint to endpoint) and delivers status update messages like typing, ringing, accepted, etc.

Requirements:

  • Dual processor, dual core 2.6GHz+ processor
  • 2 x 18GB HDD
  • 2GB RAM
  • Gigabit NIC
  • Windows Server 2003 SP1+*

Directors

Directors have two primary responsibilities: user authentication and direction.  User authentication comes in especially handy when dealing with remote users.  Because remote users cannot be redirected to their home server/pool, the director role proxies their connection through to the correct server/pool.  However, user authentication by a director is also important when an enterprise hosts multiple standard servers or enterprise pools.  Users are “homed” to a server/pool, meaning that that server/pool is the one that stores the users account information.  A director will get the user to the right pool on the first try rather than repeatedly polling every server/pool to see if the user’s account is stored there.  In summary, directors are always recommended when you have remote users and/or multiple standard servers or enterprise pools.

Conferencing Servers

Conferencing servers, in contrast to front-end servers, are designed to maintain a central hub for communications when more than two parties are involved.  This minimizes bandwidth requirements for the conference by maintaining one open channel per endpoint.  Conferencing servers also register and update “focus” on front-end servers.  “Focus” means security and state management for conferences.  Conferencing servers come in two major flavors: A/V conferencing servers, which facilitate audio and/or video conferences, and Web conferencing servers, which facilitate Live Meeting 2007 conferences.

The IM conferencing server:

  • Controls focus for three or more IM participants
  • Focus includes:
    • Participant list tracking
    • Leader determination
    • Security and management controls
  • Is installed by default on all front-end servers and cannot be installed separately

The A/V conferencing server:

  • Enables three or more parties to have audio and/or video chats (two parties use a point-to-point connection)
  • Reduces bandwidth requirements by mixing audio from other participants into one channel instead of n channels

The Web conferencing server:

  • Allows application and file sharing
  • May leverage an A/V conferencing server to distribute audio/video in a Live Meeting

The Telephony conferencing server:

  • Provides functionality for exposing an audio conference to the PSTN
  • Allows the organizer to
    • Mute everyone except the presenter
    • Mute themselves
    • Remove parties
  • Is installed by default on all front-end servers and cannot be installed separately

Requirements**:

  • Dual processor, dual core 3.0GHz+ processor
  • 2 x 18GB HDD
  • 4GB+ RAM
  • Gigabit NIC
  • Windows Server 2003 SP1+*

Archiving and CDR Server

The archiving and CDR server is designed to provide a first-shot at archiving instant messaging sessions and call detail records for compliance purposes.  Although this is Microsoft’s stated intent for the role, it’s hard for a layman to imagine how an out-of-the-box solution could really meet the compliance needs of any reasonable organization.  I have not been able to locate any customizable fields, which means that we have a great track record of who called whom at what time, but that’s pretty much it.

Requirements:

  • Dual processor, dual core 2.6GHz+ processor
  • 2 x 18GB HDD
  • 4GB+ (16GB+ if archiving is enabled) RAM
  • Gigabit NIC
  • Windows Server 2003 SP1+*

* I was not able get the OCS primary installer to run successfully on Windows Server 2008 RTM.  It may be that the individual installers would run successfully, but I have not confirmed this.  The only role I have successfully installed on Windows Server 2008 is Speech Server 2007.

** The work of mixing audio channels is intense; A/V conferencing servers will benefit from more robust hardware.

How to: Manage a successful skunk works project

My recent experience with discovering, evaluating, and evangelizing Office Communications Server at Extend Health has been great in many senses.  Many projects fail or are axed in their infancy stages.  Some projects probably deserve to be axed at that stage – one of the primary principles of brainstorming is that you entertain all ideas, even the bad ones.  But what about the projects that shouldn’t fail at that stage?  What about the project that are simply struggling to get off the ground and deserve a credible pilot?

I would contest that many projects fail in the idea/skunk works phase because of management failures.  I’m sure many managers of skunk works projects have the best intentions and are qualified managers, but managing a skunk works project is different than managing a sanctioned project.  I scoured the Web (after the project had moved into pilot phase) to see whether there were any documented best practices for managing this type of project.  I didn’t find any distilled, coherent best practices repositories, but I did find some good material that should be referenced here.

Martin Tate had the most to contribute as far as I’m concerned.  He posted a presentation called Implementing Best Practice – A Different Approach, Using ‘Skunkworks’.  In this presentation, he details what skunk works projects are and some of the classic examples (including 3M’s Post-it Notes, Audi’s Quattro, DuPont’s Kevlar, Ford’s Mustang, IBM’s first PC, and the original, Lockheed Martin’s SR-71).  [Note: there weren’t any citations, so I’d encourage you to do your own research on those examples.]  He also included several bullet points that just happened to be true of the project I just completed, which was probably dumb luck on my part.  He says that skunk works projects should:

  • Like the customer
  • Start small and build up
  • Maintain low visibility
  • Find senior sponsor
  • Manage the approvals process
  • Create team from the right people

Judy Artunian also had some excellent input in her article for ComputerWorld entitled Skunk Works: The Sweet Smell of Success.  She adds these recommendations:

  • Handful of employees with knack for taking a fresh look
  • Keep the operation under wraps
  • Demonstrate alignment with business objectives
  • Keep tabs, but don’t impose a schedule on the project

Finally, Ken Smith blogged about skunk works projects and said:

  • Don’t allow skunk works projects to delay normal projects
  • Check with product & project/product management first
  • Be ready for disappointment

I think the two things that really went right for this project were the fact that I had the CTO as my corporate sponsor, and the fact that I released information at a controlled pace.  I firmly believe that if I had disseminated the information as rapidly as I gathered it, there would have been an overwhelming and sometimes inaccurate flow of information out of the project.  Therefore, if I were to list my own bullet point best practices for skunk works project management (including rehashed versions of points above), this would be it in order of decreasing priority*:

  1. Think like the customer (even if the customer is internal)
  2. Demonstrate alignment with business objectives
  3. Don’t volunteer to manage a project you don’t believe in
  4. Find a corporate sponsor who believes in the project (or at least understands its importance and believes in you)
  5. Define a series of informal milestones to serve as gating factors for going forward
  6. Maintain low visibility
  7. Disseminate information at a controlled pace
  8. Keep in mind that only a very small percentage of skunk works projects succeed; balance cost-benefit ratio constantly

* If your project doesn’t take into account the first four objectives, don’t bother.  Scrap it and move onto something with more business value.