Embracing Change: Lessons from an Immigrant

6 Aug

By Milena Higgins, Founder and Innovation Coach, Audacious Legal

Reposted with the author’s permission. Ed.

I was a 12 year old in Poland when the communist regime was in full force. People could not openly speak their minds for fear of being caught not obeying the communist rule. My parents would often meet with their friends in private, carefully selecting participants to avoid communist sympathizers, to discuss their political views and concerns for the country. Meetings like that is how an underground social movement called Solidarity got its roots in my home town of Gdansk.

I still vividly remember passing by the striking shipyard workers behind a tall metal fence with their families and supporters just on the other side. A strike in a communist country was a rare sight indeed. I felt proud of my people for taking a stand, but also felt afraid for their safety. Years later, the Solidarity movement eventually led to the fall of communism, but at the time the country was going through much political upheaval. The future was uncertain.

The winter of 1981, my mother decided to make a bold move and flee the country with my younger sister, leaving me and my dad behind. Somehow, we would figure out how to join her later. But things were not easy during this transition. Between 12 and 14, I had to adjust to dramatic changes, ultimately living with my grandmother in a city away from my home, school, and friends. After a long, turbulent two years, including a separation from my dad when Poland declared a police state with martial law, I finally re-joined my family in the suburbs of Chicago to start a new life.

In order to get through those turbulent two years, I had to learn how to embrace change. Multiple times. That experience changed me forever and influenced who I am today. I view change as a positive force that propels us forward. And leads to better things. The pace of change in the legal market is accelerating, yet lawyers in general have a hard time adjusting to change. Here are five strategies I have honed that will help you embrace change and lead your organization into the future.

  1. Focus on the ultimate goal. For me the ultimate goal was a new life in a new country. I didn’t know which country we would end up in, but I knew that wherever it was it would be better than communist Poland. I always kept that as my guide post, no matter how hard things got. Whenever anyone starts a new project or initiative, there is always a goal, a positive outcome that we expect as a result of the change. The more you focus on that result, the easier the change becomes. If you keep thinking about the old way of doing something, that will only get you to gripes and complaints about the new way. If you keep reminding people that the reason you are doing X is so that your organization can be better at Y, which will ultimately lead to Z, people start feeling good about the changes they are being asked to make. Even if the changes are temporarily painful, the ultimate goal makes the pain worth it.
  2. Measure and report milestones along the way. My family’s journey was full of unexpected twists and turns, but each twist and each turn, brought us closer to the ultimate goal. If your change effort is a three-year process that involves multiple steps, be sure to stop along the way and look back at the progress you’ve made. I advocate measuring your progress before, during, and after a change. While you’re in the midst of a large change initiative, it may feel like all you’ve experienced is pain, but when you actually look at the metrics you might discover you are actually making progress. Don’t forget to celebrate those incremental wins. Shout them out to your team. Celebrate them! They will give you the fuel to keep going.
  3. Don’t forget the silver lining. It sucked for me to be stuck in war-state Poland with my parents in another country, but having some newly-found freedom and reduced constraints as a teenager was not such a bad thing. In any change situation, there are bound to be negative side effects. But usually nestled among them are some good things. It might take a bit of extra effort to see them at first, but when you stop to think about what is going on, you can usually find them. Seek them out and shine a light on them.
  4. Look for role models. We knew of other families who made similar journeys before us. You may have done another change project five years ago. And you may remember the pain and the reward. Remind yourself and your team about that success. Or you may have a colleague or a competitor who implemented a new initiative that turned out well for them. Look to those examples for inspiration and a reason to keep going.
  5. Keep a positive attitude. I am an optimist. I see the glass as half full. At the end of the day, it all boils down to your attitude. If you are leading the change effort and you are the one grumbling and complaining about it, be sure that your people will follow your lead. If you are positive they will be positive too. And that circles right back to point #1— focus on the ultimate goal. There is a reason you decided to make a change in the first place. That reason is positive. If it weren’t you wouldn’t have started going down this path. So brush those Negative Nellies and Debbie Downers aside and keep on marching forward.

Change is a good thing. Really. Use these strategies to guide your law firm, legal department, or legal team through change to new or improved legal services. Change may be temporarily painful but it ultimately leads to better things. And all of our clients want that!

Advertisements

Changing Knowledge Management (KM) Priorities

21 Jul

By Ron Friedmann, Partner, Fireman & Company

How and why are KM priorities changing? This was the topic of an ILTA virtual roundtable I moderated on July 6, 2017. I report here on the survey administered in advance and the flavor of the discussion (as moderator, I was not able to take good notes). An appendix contains survey results we did not have time to discuss.  The panelists were:

KM Priority Survey

We opened with poll results on KM priorities. The poll was completed by 39 of 115 registrants; no demographic breakdown is available. The KM priority question offered eight options and required force ranking. Here are the average scores:

Figure 1

The results surprised me. The average priorities, except for PSLs, seemed close together. So I charted just the top three priorities:

Figure 2

While this view provides more detail, the highest priorities are split pretty evenly (except PSLs ).

What Drives KM Priorities in Firms?

In the Roundtable, rather than try teasing out the data, we focused on how firms decide on KM priorities. While some look at metrics such as usage, the general sense was that proving ROI or finding a dispositive metric remains challenging. Some focus on what clients are asking for, others on what specific practice groups want.

From my consulting, rather than the poll or discussion, I know that some firms set priorities based on formal or informal benchmarks. For example, “Our Intranet is ten years old and everyone complains about it. Maybe it’s time to fix it.” Or “Most firms have enterprise search;, maybe we need to get it too.”

A discussion about KM drivers aligns well with a recent industry development, the rise of corporate law department legal operations professionals.

Align with CLOC?

The Corporate Legal Operations Consortium (CLOC) had its annual conference in May and I knew that panelist Camille Reynolds of Fenwick & West attended. She reported on her impressions, which include that CLOC members are very interested in KM, improving practice efficiency, and establishing better workflows with outside counsel.

Jason Barnwell of Microsoft represents the legal operations view of a very large and forward thinking client. He reports that his law department’s challenge is managing all incoming work product from multiple outside counsel. Microsoft would love a single platform to consolidate work product.

I agree that a single platform would be helpful, but aligning multiple clients and law firms faces challenges. I commented that I had blogged in 2003 about the rise of extranets and, even then, noted that multiple law firms would not want to use multiple extranets. Instead, I suggested in that post that the market needs a data exchange standard. I am not sure that would be any easier though.

We then discussed the potential alignment of law firm KM professionals with in-house legal operations professionals. At least among this group, there was consensus that it would make a natural alliance: both want to improve efficiency and are open to new workflows and tools that would help do so.

 Artificial Intelligence (AI): The Holy Grail?

You cannot discuss KM priorities today without including AI. We started this portion of the roundtable by sharing the survey results:

Figure 3

These results mirror some other AI surveys I have seen and group AI discussions I have participated in: more talk and consideration than deployment. (I note, not from this discussion, that a recent Altman Weil survey suggests more activity than this, at least in the largest firms.)

The panelists agreed that the clearest use case for legal AI today is machine learning to accelerate due diligence review. Eagerness to investigate AI remains high, but no one has yet found the killer app.

We discussed that “big data” in legal is not nearly as big as in other markets, but pointed out there is nonetheless enough for interesting applications of AI.

Conclusions

Anyone who expected a clear roadmap to emerge from the discussion left disappointed. But for those who wanted to understand where the action is and how to think about setting priorities left with some good information and tips.

Appendix – Poll Results We Did Not Discuss

I hate wasting interesting poll results so I include here, without discussion, two additional poll question results that did not have time to cover .

The poll asked respondents about three 2017 or 2018 major projects many firms are undertaking. Only one choice was allowed:

Figure 4

Another common KM discussion topic is client-facing KM. We asked about where firms are on that :

Figure 5

 

A New Era for Managing and Finding Experience in Large Law Firms

9 Jun

By Ron Friedmann, Partner, Fireman & Company

Experience management – the ability to find relevant matters and lawyers with specific experience – has become critical for both knowledge management (KM) and marketing large law firms. Here, I explain here its importance and examine the evolution of the technology and processes that support it.

KM Evolves from Documents to Experience

Lawyers have long tried to find and re-use work product. They started in the paper days with file folders and indexes. With the rise of PCs and networks in the early 1990s, forward thinking firms deployed full-text search tools to find work product. Variations on search continued for over a dozen years.

Search started with promise, but is ending with disappointment. It was not smart enough, often failing to find relevant documents. Even when lawyers did find good documents, those documents had limited re-use value without context about the matter or from the author. This dramatically changed KM around 2005, when new technology emerged that tapped into matter types and billing data (number of hours and narratives) to infer lawyer experience and a matter’s area of law. With this new capability, KM professionals learned that finding relevant lawyers and matters was at least as valuable as finding documents, if not more so. A quick conversation with an experienced lawyer not only supplied context, but also identified the most relevant documents. The software also significantly improved relevance ranking in document searches.

Need for Experience Data Expands

The 2008 economic crisis spawned many changes in large law firms. Marketing and business development grew in importance and firms hired pricing professionals to set budgets and alternative fees. Management started analyzing profitability by matter, client, partner, and type of matter. These new initiatives require accurate information about lawyers’ experience and the matter’s area of law:

  • Winning Pitches. Clients want lawyers with proven expertise to solve their problems. Proving expertise, whether in formal written proposals or informal discussions, requires a dossier that demonstrates the firm’s relevant experience and best- suited lawyers.
  • Establishing Expertise Publicly. To win opportunities to pitch, firms must establish their expertise publicly by presenting specific matter experience by practice, earning league table top rankings, and winning awards. All three require locating relevant experience.
  • Pricing. Pricing professionals need to find similar matters to estimate costs and set prices, which requires an accurate record of matter type and experience.
  • Integrating Laterals and Cross-Selling. With lawyers regularly moving laterally to new firms, the complexion of cross-selling has changed. Personal connections and memory of past matters no longer suffices. To cross-sell effectively, partners need a constantly refreshed source of information on matters and lawyers.  

Approaches to Managing Experience Evolve

Law firms have tried many ways to capture and manage experience, most with limited success. Today, new and much better options exist.

Email was the earliest approach with the all too familiar “pardon the interruption” (PTI) messages that lawyers send to internal mailing lists. In some firms, this yields answers but they come at a cost: PTIs interrupt the other lawyers and several may answer, duplicating effort. Because firms rarely capture the responses, lawyers ask the same questions again. In some firms, the culture does not tolerate this practice.

Early in the century, many firms created area-of-law taxonomies to catalog expertise. Firms asked lawyers to rate themselves, which generally failed because many lawyers never rated themselves and those who did often under- or over-rated their experience. Self-rating has a place, but not as the primary way to capture experience.

The inference approach described above was a big breakthrough and served firms fairly well for about a decade. Then, requirements changed and its limitations became apparent. Inference captures only area of law. For pitches, pricing, and profitability, more detail is needed. Many firms created custom databases to capture deal attributes or litigation metadata, such as opposing party, judge, and jurisdiction. Firms also licensed specialized RFP generation software. Unfortunately, these approaches require substantial care and feeding and have their own significant technology or process limitations.

Rise of Modern Tools and Processes

Recently, a new class of software has come on the market designed to manage experience. These tools collect important details about lawyers and matters, offer flexible reporting, integrate with other law firm systems, and have simple-to-use interfaces. Furthermore, they offer a single, enterprise system that can power marketing, KM, finance, and other functions.

However, software alone is not enough; someone must populate the data. Reluctance to hire staff to do this has fallen as firms respond to the need to pitch, price, and analyze profitability. Many Marketing Departments already invest heavily in capturing this type of data. Finance and new business intakes often contribute and KM departments happily contribute given that they can ride on the experience system’s coattails.

These systems rest on more accurate matter typing. Depending on the nature of the legal work, accurately characterizing matters may be possible only well into the life of the matter. Marketing or KM typically chases down the information at suitable points in the matter, which can be days, weeks, or months after inception. Some leading firms, however, are building tools based on role and time entry with phase and task codes as triggers that add a controlled workflow component to capturing information over the lifecycle of a matter. That lifecycle integration holds real promise.

As a Fireman & Company partner, I have worked with several leading providers of experience management systems, including Prosperoware, Neudesic Firm Directory, and Foundation Software Group. Each offers its own spin on experience management. Regular readers know that I do not review products or even compare and contrast them. I mention brands to illustrate the general point – and to disclose business relationships. Fireman & Company’s partnership with Foundation Software Group, announced on May 31 (here), is the most recent partnership with a leading legal tech provider. As with all of Fireman & Company partnerships, we gained experience with the platform through our clients.

Through Foundation (and we can extend this thinking to experience management information in general), we began to see the exponential value of experience data when run through machine learning and AI tools and combined with document management, time narratives and entries, and other content. Auto-classification, analytics and entity (including clause) extraction are more dependable and effective when trustworthy experience data is available. As we move forward into an integrated information environment, experience management has become a core component of a firm’s service delivery infrastructure.

Quoth the RAVN: How iManage’s RAVN Acquisition Will Affect the Legal Market

25 May

By Joshua Fireman and Ron Friedmann, Fireman & Company 

The document management, search and AI spaces have seen accelerated activity and innovation over the past two years. iManage’s acquisition of RAVN, announced May 24, 2017, brings these three streams together. Integrating all three categories into a single product likely will have a significant impact well beyond the deal. We discuss here the potential impact on document management (DM), search and artificial intelligence (AI), and how this new platform approach validates both emerging and maturing trends. For context, we start with an overview of how the market arrived at this point.

A Decade of Change: Trends in Legal Tech

After years of relative inactivity on the vendor side in both search and document management, we now see a generational, if not a once-in-career, shift in the market. Many factors have led to this major point of inflection.

  • Lawyer needs have changed significantly versus a decade ago, with better collaboration, real (not just stated) information governance, more value for clients, mobile work (from anywhere), and cybersecurity.
  • Technology has changed dramatically with the rise of cloud computing, mobile devices and apps, and enterprise-quality open source code.
  • Artificial intelligence needs to be considered separately. The rise of machine learning and other AI creates many opportunities. While the AI hype concerns us, we do see tremendous potential.
  • Client demands have increased. General Counsel want more efficiency, more security, and more transparency in how their outside counsel work. With 1000 people attending the CLOC conference earlier this month, it is clear clients are organizing and improving their game.

Accelerated Market Innovation in DM and Search

Prompted by these changes, we have seen major advances in the two leading providers of document management systems.  As a partner of iManage and NetDocuments, Fireman & Company has extensive experience with both and here is what we see.  NetDocuments, which started as a cloud-based system, has gained significant market share in the last few years as it has upgraded its features, functionality, and back-end architecture. iManage, now back to a stand-alone company with the ability to invest and plan its own future, has invested significantly in developing the next generation of its flagship product.

We have also seen substantial change in the search market. Recommind, long the dominant provider of enterprise search for law firm KM, is rapidly losing market share. Multiple new or newly improved options now vie for mind and market share. We see that most firms buying or replacing existing search installation are looking at Handshake Universal Search and RAVN (again, Fireman & Company partners with both). But other choices are available, including the industrial-strength Sinequa.

The Acquisition’s Impact on DM, Search and AI

So, with all the changes, what should we make of iManage’s acquisition of RAVN? As noted above, this is a bold move. It brings together DM, search and AI in ways that may that well affect all three. What this means for law firms, both those comfortable with their current tech stack and those evaluating significant changes, is important to consider:

Document Management

Foundational Platform. Both iManage and NetDocuments have been innovating at a rapid pace over the past few years. Both companies have invested in advanced data analytics, entity extraction and other services that add significant value to matter management, pricing analysis and knowledge management. What RAVN brings to iManage is a powerful analytics, machine learning and visualization platform, in addition to solid enterprise search capabilities. With iManage planning to integrate key RAVN tools (auto classification and clause banks, for example) into its platform, iManage is doubling down on a vision of the DMS as a legal service delivery foundation. NetDocuments has also been developing similar capabilities with its sizable investment in SOLR and Lucene open source search tools.

Adoption. We discuss AI in a separate section below but, for DM purposes, we expect that with the addition of AI and other extraction / classification techniques, adoption will grow. (As a side note, we see many law firms across size ranges where lawyer use of the DM ranges from under 50% to typically no higher than 75%.) With the potential for automatically classifying document type and extracting key metadata, the effort involved in saving documents may go down. And, finding documents in the DM – even without search – likely will be easier with more fields and facets available to filter content quickly.

Information Governance. In the age of regular and massive cyber breaches, every law firm now takes cyber and data security seriously. Beyond hardening the perimeter and detecting intrusions, today most firms assume that at some point a breach will occur. To minimize damage when an intruder gains access, documents must be properly secured to those who need access. Consequently, firms are rapidly moving to gain wider adoption of DM and better lawyer compliance with information governance policies. Gaining that compliance remains a challenge because it requires lawyers to take extra manual, tedious steps. The addition of AI for automated entity extraction and document classification will almost certainly ease IG challenges.

Search

Search as Platform. Legal-focused search platforms have evolved significantly over the past two years. We see the RAVN acquisition as validating the “search as a platform” approach that RAVN, Handshake Universal Search and Sinequa had adopted. Search is no longer a black box bolt-on tool to other law firm IT infrastructure. Each of the above platforms offers APIs and developer tools to build search apps, leverage personas and deliver information in context based on client work, time entry and more.

Almost all firms have DM, but only some have enterprise class search. The iManage acquisition of RAVN signals that search has now become a core part of DM. (NetDocuments has also made search core to its platform by using SOLR and LUCENE open source search tools.) We believe the integration of search into the DM and the addition of AI to search will eliminate many of the past challenges with search, such as auto-classification and relevance ranking.

Artificial Intelligence

Mainstreaming Legal AI. As noted at the outset, we have observed a bit too much hype around legal AI for our taste. But, we do believe that the acquisition of RAVN signals the validation of legal AI. As important, it reflects a more complete “productization” of AI. Most legal AI products to date tend to be stand-alone solutions. With AI integrated into a core platform, it will become a tool lawyers use regularly. That is, instead of “going to an AI tool” for a specific task, the “AI tool will come to lawyers” as part of tools and workflows already in place.

Competitive Hotbed. NetDocuments has told us that they too have looked at AI companies to acquire and, in parallel, is considering a build rather than buy strategy. With Google, Facebook, and Amazon all having open sourced their AI tools, building AI into other systems at relatively low cost and effort is now possible (compared to what it would have taken even two years ago).  iManage’s RAVN acquisition will likely push improvements in this area to even more law firms, lawyers and clients. Beyond DM, we think this acquisition will accelerate the already vibrant development of use-case specific AI tools. The list of AI tools grows longer by the month. Prominent acquisitions are always a good motivator for budding entrepreneurs. Moreover, so many types of AI and potential legal market use cases for it exist that we think other products will continue to thrive.

In Sum

Regardless of the deal’s details, we see this as a turning point for the market. Document management will move from what had been almost an afterthought utility at many firms to being deeply integrated with the business and practice of law. Likewise, both search and AI will become core parts of law firm infrastructure.

We have long focused on identifying and understanding law practice and law firm business problems and then turned to technology as part of the solution. For us, this deal heralds two exciting advances in solution design. First, many good solutions will now be easier to design and deploy. And second, the deal’s validation increases the awareness and likely willingness of lawyers and firms to embrace more sophisticated solutions.

How to Survive Complex Projects, Part 6: Countdown to Launch and the Rule of Seven

3 May

By Andrea Alliston, Partner, Knowledge Management, and Nicola Shaver, Director of Knowledge Management, Stikeman Elliott LLP

In part 5 of this series, the wonderful world of user acceptance testing (UAT) was explored. Ed.

Now, you have tested your platform and you are satisfied that you have a product ready to release to your users. How do you manage the launch?

Get the message right

An old adage of marketing, the Rule of Seven, proclaims that prospective buyers need to see or hear the product’s marketing message at least seven times before they will be persuaded to buy it. While some argue that the rule is outdated in today’s digital world, the message behind it remains unarguably valid: to succeed, a marketing effort must be repetitive and consistent. This maxim applies equally to prospective buyers of new products in the world at large and prospective users of a new platform inside an organization or firm. Absent repetition, buyers and users will not recognize – and therefore not trust – the product on offer. Running a few advertisements or sending out a few communications and never speaking about the product again simply will not work. Only continuous effort ensures success.

Repeated, consistent promotion was central to our launch and training plan during our enterprise search project and it was broadly successful. In a global firm with eight offices and several thousand people, we wanted to be certain that no one could say they had not heard of our new platform by the end of our launch campaign.

Vary the format

We also kept our audience at the forefront of our minds. Knowing that lawyers get massive volumes of email, we understood that they would not read unnecessary messages. How then could we ensure that each and every person in the firm would be touched seven times by our message without overwhelming – or worse, annoying – them? Our answer was variety or, in marketing-speak, we took a multi-channel approach. People are less likely to protest if the message is received in different ways, at different times. So we planned to rely on all of the following:

  • Email
  • Intranet posts
  • Telephone displays
  • Presentations at practice group meetings
  • Promotional posters
  • Launch video
  • Branded swag
  • Treats
  • Elevator chat

We tried to think of every audio, visual, and tactile method we could use to deliver our message.

Plan, plan, plan!

The combination of our intended quantity and variety of messaging meant that, as with all other parts of the project, being prepared was essential. We had our strategy, now we needed a plan. Months before our pre-launch communications rolled out we developed a launch plan detailing the timing, content, and format for each communication. We captured the lead time needed for ordering our swag and for scripting, shooting, and polishing our video. Like any other project plan, each task was assigned to a specific person and checked off as completed.

Figure 1. Map for our search project

Be prepared for setbacks

A launch and training campaign informed by a well-considered strategy and supported by a detailed plan, is primed for success. Nevertheless, hiccups inevitably will arise along the way. During our search project, we developed a bold plan for our promotional video to play automatically as soon as people logged on to their computers on launch day. This was more complicated than we had anticipated. Many people do not log off their computers before leaving the office each day, so the video play was unintentionally staggered (in spite of an email circulated asking everyone log off the night before). And, we had no way to thoroughly test whether the automatic video-play would work on everyone’s computers given the variance in individual set-up. Many people were surprised by the video when it suddenly played a day or two late, and some complained that when the video started playing they were not sure what it was or how to get rid of it. Our intentions were admirable, but the execution taught us that something that cannot be tested properly beforehand should not be part of the launch plan. Needless to say, we would not do this again.

Train your users

Like our communication strategy, our training strategy rested on offering variety. We knew that partners in our large offices would not sit through an hour-long session on search. We also knew that our assistants prefer formal training over less-structured training. In the end, we offered formal training sessions geared to administrative people, drop-in sessions for everyone (but mainly geared to associates), and one-on-one training for partners. We engaged our IT training team for administrative people, but trained the lawyers ourselves. In our smaller offices, we leveraged lunch-and-learns targeted at everyone in the office.

You will know and understand your own firm well enough to choose the kind of training that works best for your user groups. Accept that one kind of training may not work for everyone and build flexibility into your approach.

Lessons Learned

Promotion never ends: We discovered that the launch and training phase of a project never ends. Six months after launch, we were rolling out new training courses, planning “Summer Camps,” and offering non-traditional learning opportunities with games like Scavenger Hunts to tap into lawyers’ natural competitiveness. If you are promoting a tool internally, your efforts will need to be repetitive and continuous.

You will never reach everyone: Although we likely did touch everyone in the firm during our promotional outreach, we still come across lawyers who become excited when we show them subtleties of our search platform and eagerly suggest we promote and teach people about it. Our efforts passed some people by and this taught us that work always remains to be done. On the other hand, keeping yourself from getting discouraged is important – if you have properly implemented a tool that works, you should be able to see from usage statistics that people are using the platform. Our enterprise search statistics are very healthy and this keeps us motivated.

Celebrate: You have done it. Your complex project is finished and you have released a new tool to your users. Take a moment for celebration and acknowledge the help from everyone on the project team. After our enterprise search project, we held a celebratory dinner and gave out spirit awards that included “Best Facial Expressions” and “Most Patience” during the project. The team will appreciate the thought and this will help everyone re-group for the next big project.

How to Survive Complex Projects, Part 5: Getting Your Testing Right

20 Apr

By Andrea Alliston, Partner, Knowledge Management, and Nicola Shaver, Director of Knowledge Management, Stikeman Elliott LLP

In part 4 of this series, the authors guided us through implementation and development. Ed.

Once your platform has been implemented and handed over to your team, you will need to make sure that everything works the way you want it to work. After all of the effort and time you have put in, the last thing you want is to fail now. This is where testing comes into play.

What is User Acceptance Testing?

Unfortunately, we know from experience that if KM does not conduct the testing, you can never be certain the product you hand over to your users will work as they expect it to. This is true regardless of how comprehensive the IT department’s quality assurance testing is. The hard truth is that IT professionals see things differently than most users. IT people are not lawyers and do not think like lawyers. As a KM professional, your job is to operate between the technical and legal realms to evaluate whether your users will be able to incorporate into their daily practice the new platform as designed. This crucial step is called user acceptance testing (UAT).

The form and quantity of UAT the KM team performs varies enormously from firm to firm and project to project. We have worked in firms where very little testing is done before launch and in firms where massive testing is done. In truth, the proof is in the pudding – the more effort you put into UAT, the better your outcome will be.

Where do you start?

Our first step in UAT always is to create a test plan. We find this so indispensable that we now create plans even for testing the most minor upgrades to a platform. When creating the test plan, return to your requirements and use cases developed for your business case. You should plan to test every element of functionality a user would touch, understanding the desired outcome based on your requirements. For example, in our search system we determined that ensuring that a filter worked when applied to a set of search results would not be sufficient. Instead, we also needed to check that the list of values for each filter was correct, we could both exclude and apply filter values, the “or” (rather than “and”) logic of the filter values was in place when multiple filter values were applied at once, and we could both scroll through and search for filter values.

Our test plans usually outline a process where each team member mimics a user’s journey through the platform based on our use cases. Depending on what we are testing, we may allocate roles to different testers, with everyone following a slightly different journey through the platform. For example, partners likely use most platforms differently than legal assistants and you need to ensure the functionality works equally well for both. Role-based testing helps you to confirm that your new technology is ready to be released to everyone in the firm.

People doing the testing should note failures in not only pure functionality, but also design. An IT professional testing a grid view may feel that clicking on a column heading and having the results sort is just fine. But, a KM professional will likely recognize that unless a symbol indicates that sort functionality is available, most users will not know the option exists. Your requirements likely included this need early on, but testers should be on the look-out for its having made its way into the actual build.

Because our firm is Canadian and has a major office in French-speaking Quebec, testing bilingual functionality is vital. Recognize and prepare for the fact that your firm may have its own unique jurisdictional requirements or other complexities that must be addressed through a customized UAT process.

Allow sufficient time for UAT

Depending on the complexity of your project and testing, UAT may take some time. In our experience, no test phase ever goes without hitches. Assume you will have multiple issues to report and some elements of the functionality will not work properly on first review. Also prepare for substantial back and forth with IT and the vendor as you work through the issues that testing reveals. During our enterprise search project, we completed several rounds of testing for each component of the project.

Improve inter-departmental communication

Perhaps the most important takeaway from our major projects has been the importance of communication. Communication is critical during testing and it is hard. Communication among team members and between KM and IT, the firm, and external vendor all are critical. Most communication challenges that arise between your team and IT likely can be improved through mutual empathy and understanding.

As observed, most IT professionals speak a different language than legal professionals and you must account for this in the communication style you adopt between departments. For example, during testing, it is tempting for individuals who find major issues to want to report them to IT at once to be addressed quickly. Of course, in some instances this may well be appropriate. But, we have found that waiting to report all issues together generally works better. One of our team members collects feedback from everyone else, consolidates it into a single spreadsheet, and sends it to IT. Every piece of feedback is numbered and, wherever possible, accompanied by screenshots. We have found that the words we used to describe the problems we find often are insufficient to communicate those issues to IT. In fact, we have been tempted to sit beside our IT colleagues and show them what we have experienced on our screens. This rarely is the best use of anyone’s time, nor should it be necessary as visual props should abate confusion and frustration.

Lessons Learned

Engage multiple testers: People have different aptitudes for testing and technical detail. In the same testing round, we have had someone report that everything looks great, while another person reports over 25 issues. So, you will need a team of testers for UAT. You simply cannot do this on your own.

Understand your team’s limitations: During our enterprise search project, the KM team tested the system’s ethical walls compliance. Even though we managed to do it, it was needlessly time-consuming and, in retrospect, this should more properly have been handled by IT. We would not take this on ourselves again. So be realistic – some things are better left to IT.

Know when to stop: After you have signed-off on a component of the project, stop testing that component’s functionality. It sounds simple, but letting go and trusting that the fixes really are in place can be difficult in practice.  Now, if in using the platform you stumble across new issues, these of course must be reported. Otherwise, encourage your team to trust your firm’s IT and the vendor and move on to the project’s next stage.

Coming up in part 6: In our next post, we get to the exciting part – how to prepare for launch and take-off.

How to Survive Complex Projects, Part 4: A Marathon, Not a Sprint

5 Apr

By Andrea Alliston, Partner, Knowledge Management, and Nicola Shaver, Director of Knowledge Management, Stikeman Elliott LLP

In part 3 of the series, the authors walked us through the key steps in planning complex projects. Ed.

With a plan of attack in place, it finally is time to start on development and implementation. The complexity of your project will depend on several factors; but, no matter whether you are implementing an out-of-the-box or highly customized solution, substantial KM technology projects usually involve an intensive period of technology development.   Enterprise search – aggregating information from multiple sources and presenting it in a user friendly way – is inherently complex.  So, we knew that implementing our project would not be easy.

KM Involvement in Development and Implementation

Even with an out-of-the box solution, you will have options for how different aspects of your project can be implemented. This becomes more complex if you wish to customize the solution.  In our case, we increased the complexity by deciding to implement a new taxonomy tool and KM content management tool in parallel with our search implementation.

In the throes of a complex project, forgetting what you have decided and why is easy to do.   Combined with developers bombarding you with questions regarding the functionality you want, this can quickly overwhelm you.  As observed in an earlier post, having documented detailed requirements was our saving grace in these situations again and again.

A difficult reality in development and implementation is making compromises. Even with the most careful planning, nothing goes perfectly and being told that the design you have envisioned cannot be achieved is frustrating.  What do you do?  You go back to your design principles.  You may recall that in our search project our overarching theme was a relentless focus on user experience.  So, we always chose options that would best support the user experience, even if those options were not perfect.

Data Integration

Because very few KM projects involve stand-alone technologies, a heightened demand for tighter integration across multiple platforms and data integration become key aspects of most projects.   In our implementation, we integrated as many data sources as possible into search, which proved critical to improving relevancy. Those data sources included our InterAction contact management platform, Aderant financial system, matters and experience database, website, blogs, document management system, and KM content management system.

But, integrating data from multiple sources brings its own challenges. In our case, data was sometimes populated from the wrong system or from the wrong field in the right system, which required a bit of detective work to sort out. Another challenge was working with the owners of some of the data to address inconsistencies that the search project revealed.   Enterprise search and other technology projects cast a giant magnifying glass over your data (and its warts). You need to be prepared to address issues that might arise.

Lessons Learned

Manage expectations:  Highly visible, significant technology projects often lead to high expectations.  Throughout our development stage, we managed stakeholders’ expectations with regular updates and early demonstrations.

Tune and tweak: Do not be afraid to spend time making small tweaks that have a big impact.  One of the most valuable efforts we put into our project was fine-tuning relevancy.  Although taking the time mid-implementation to focus on relevancy was difficult, doing so significantly improved the user experience.

Master your metadata:  One reason we were able to identify incorrect data was that we had prepared a detailed data map listing all of the metadata appearing in the search application, including what it should look like and where it was expected to come from.  We revisited this (and other) documentation throughout the development stage.

Coming up in part 5: In our next post, we look at the role KM can and should play in testing complex KM technology projects.