Focus Groups and Workshops

Focus groups and workshops are separate techniques that involve getting a group of people together for a purpose, but the emphasis in each case is different. A focus group is generally intended to gather people’s reactions to a specific product, service, or situation, while a workshop may be convened for a wider variety of reasons.

Both techniques define an objective, establish a place, time, and procedures to be used, identify participants and roles (sponsor, moderator or facilitator, scribe, SME or other participant, and possibly timekeeper), conduct the discussions and other work, document the results, and follow up on any identified communications and action items. In both cases the organization will (presumably) take subsequent actions based on the insight gained.

Focus groups are often conducted to gauge the desirability or positioning of a new or existing consumer product, but this doesn’t always have to be the case.

Both techniques can gather a lot of information or accomplish other goals in a short period of time, because face-to-face interaction can enhance communication and understanding. Potential downsides include issues of trust, availability, factions, and other issues.

Posted in Tools and methods | Tagged , , | Leave a comment

TWSL Series 05: What Business Analysts Should Know About Software

Today I gave this webinar for the Tom Woods School of Life Lunch and Learn Series. The slides are here.

Posted in Tools and methods | Tagged , , , , | Leave a comment

TWSL Series 04: Engagement Phases In Detail

Today I gave this webinar for the Tom Woods School of Life Lunch and Learn Series. The slides are here.

Posted in Tools and methods | Tagged , , , | Leave a comment

Glossary

You all know what a glossary is. The BABOK struggles to find much to say about it. It is basically just a list of words and definitions used in the context of the operations being investigated. It is a good idea to create or maintain one when there a many new people in a situation, when you are working in a new industry, when you are using new solution techniques or components, or when engaged in domain knowledge acquisition generally. It is basically an aid to communication, and we know that business analysis is all about communication, especially when trying to translate between users/customers and various technical specialists.

Different terms of art tend to apply in different phases.

  • Intended Use: Problem statements are often expressed using broad and understandable language. However, organizational and engagement goals can absolutely refer to specific operations, technologies, or methods from specialized areas of practice. Moreover, the language of business, particularly with regard to financial, legal, and regulatory matters, can employ multiple terms of art.
  • Conceptual Model: This is where the language of the operation or solution being built or modified comes into play. It can involve equipment, processes, materials, operations, measurements (Canadian Standard Freeness remains the coolest measurement ever!), and even people (in the form of job descriptions).
  • Data: Data management and analysis has a certain vocabulary of its own because it is gathered in so many different ways and has so many different characteristics both singularly and en masse.
  • Requirements: Language in this phase tends to use terminology primarily defined elsewhere, although it’s good to be aware of different kinds of requirements (e.g., functional vs. non-functional, business/stakeholder/solution/transition, and so on).
  • Design: Work in this phase tends to use language defined elsewhere as well.
  • Implementation: Implementation tends to require a wide variety of specialized practitioners, each of which may use specialized verbiage. Software engineers have a large and specialized vocabulary, management frameworks, computer languages, programming paradigms, and the like, but there are many types of practitioners, ranging from pipefitters to graphic artists.
  • Test and Acceptance: There are almost limitless ways to verify and validate solutions. Software is just one field and that process has plenty of its own verbiage and even equipment and processes.
Posted in Tools and methods | Tagged , , | Leave a comment

TWSL Series 03: What Is Business Analysis?

Today I gave this webinar for the Tom Woods School of Life Lunch and Learn Series. The slides are here.

Posted in Tools and methods | Tagged , , , | Leave a comment

TWSL Series 02: Framework Introduction

Today I gave this webinar for the Tom Woods School of Life Lunch and Learn Series. The slides are here.

Posted in Tools and methods | Tagged , , , | Leave a comment

Metrics and Key Performance Indicators (KPIs)

Metrics and KPIs indicate how an operation or system is performing relative to trend or a defined goal. They may take the form of point values (ninety-five percent uptime for a manufacturing process), ranges (plus or minus one percent of target), or probabilities (less than fifteen minute wait time, eighty-five percent of the time). They can be measured over time periods of any length.

Many types of values can drive decisions, but what makes something a metric or KPI is how the decision relates to strategic organizational goals. For example, a data point associated with a document or a record may affect whether that item is diverted to one processing operation or another. That data point drives a decision (and potentially thousands of them a day), but that decision does not indicate progress toward a goal or standard. Providing service in an effective and timely manner, reducing waster, hitting quality or financial targets, and meeting or exceeding market standards are all examples of strategic assessments and can be thought of as governing metrics.

Note that values that serve as metrics and KPIs are likely to be derived from the operation of a running process or system, as shown above (at least in part), and not values that describe or govern it, as shown below. Note also that the above figure, at least in this context, refers to a working solution, while the figure below describes the engagement during which it was created or modified.

Indicators may take many forms, but should have many of the following important characteristics (item names from the BABOK, think about how this advice is similar to that for writing user stories):

  • Clear: understandable and without ambiguity
  • Relevant: germane to the activity
  • Economical: obtainable without excess time and effort
  • Adequate: embodies enough information to provide effective guidance
  • Quantifiable: can be expressed in values that can be validated
  • Trustworthy and Credible: deriving from agreed-upon, accurate, authoritative sources

It’s nice when metrics are expressed in terms of simple values, but they can become very complex and synthetic. I worked on one program that tried to improve the “readiness” of large quantities of equipment. The equipment needed a high degree of maintenance and inspection, and was considered “not ready” whenever such work was waiting to be performed or even in progress. As a result, the value of the metric (expressed as a percentage), turned out to be surprisingly low. However, small changes to that value were regarded as highly significant. The important thing is to understand the source, derivation, and context of the metric or KPI, no matter what form it takes. (I could tell you some other stories about that and other programs, but those are discussions for another day. Ask me over a coffee sometime.)

Another issue with KPIs is that they may not capture what is really wanted. For example, I recently encountered a situation where a manager established a goal of having an eighty percent “win rate” for sales encounters. Someone else asked the question of whether the metric would have the desired effect. Going back into history, industrial planners in the former Soviet Union used to set quotas for the production of nails. The problem was that the KPI was expressed in tons of nails, and this was done without consideration of how difficult different kinds of nails were to make. As a result, producers tended to manufacture nails that were easier to produce. Since thicker and wide-headed framing nails are easier to produce than finer and small-headed roofing nails, the shortages of roofing nails were so severe that buildings would be constructed all over the place — except for the roofs. This rendered the buildings unusable and they would just sit and rot in the weather.

The Soviet nail example is meant to illustrate the advantages of the price system, where the (presumably higher) price of roofing nails incentivizes producers to manufacture what is needed.

I don’t know if we always have to go as far as saying Goodhart’s Law is always applicable, but one has to be mindful of what is being measured and what it means.

Returning the “win rate” example I started with, is the real goal to ensure that sales teams get wins at a certain rate, or is it to make the most sales? Or is it to make the most sales per a given amount of effort? Or is it to bring in the most money? One possible problem with targeting the win rate alone is that tales teams may game who they try to sell to, or they may do inappropriate things (like reduce the price to buy sales, thus lowering the brand’s value). None of those are desirable.

Metrics and KPIs should be revisited and reviewed over time to ensure they represent what is wanted, and that they incentivize the desired actions. I often mention the need to be thorough, and to look at problems from multiple points of view. As always, different people with different perspectives should be consulted where it makes sense. I try to illustrate this by including the image below in many of the talks I give.

Posted in Tools and methods | Tagged , , | Leave a comment

TWSL Series 01: Simulation

Today I gave this webinar for the Tom Woods School of Life Lunch and Learn Series. The slides are here.

Posted in Tools and methods | Tagged , , , | Leave a comment

Prioritization and Backlog Management

Prioritization and backlog management are two sides of the same coin. As described in the BABOK, an important difference is that backlog management is a more specific case within the broader topic of prioritization.

The BABOK discusses prioritization in terms of business analysis information. Backlog management typically involves tasks or work items. As a practical matter, the criteria on which prioritization is carried out in both cases is the same. Considerations involve value, risk, difficulty of implementation, dependencies, and so on.

It seems to me that prioritization can involve qualities of a design or approach, determining which customers to serve first, and the order in which tasks should be worked on. While backlog items typically involve defined collections of work to be done, they are ultimately driven by the same, more general considerations.

A very important consideration in prioritization involves recognizing and making trade-offs of various kinds. Many types of trade-offs are possible, and all involve the economic way of thinking. Economics, after all, is best defined as the study of making choices under conditions of scarcity, and this involves the recognition that we always live and work under conditions of scarcity. One discussion on trade-offs is embedded in a webinar I gave on What Business Analysts Should Know About Software.

While this specific talk involved software, you can imagine that different trade-offs will be germane to other potential products and services. Considering the iPhone, for example, Apple could not produce them in every possible color. Instead, different color options (moving in a direction from conservative to increasingly bold over time) have been released only slowly, to where there remain only six color options after fifteen years. That is, the designers and managers at Apple had to look at the costs and practicality of producing devices in different colors.

Computing
• execution speed
• response time
• memory usage
• disk usage
• clarity
• modularity
• maintainability
• reusability
• abstraction
• language choice(s)
• protocols
• scalability
• uptime
• security

People and Methods
• developers
• managers
• data collectors
• analysts
• testers
• users
• customers
• speed of development
• reduction of rework
• popularity of languages and tools
• industry trends
• larger societal trends

Cost
• capital vs. ongoing
• fixed vs. variable
• licensing
• hardware
• third-party
• maintenance and support
• upgradability
• modifiability

Another classic discussion of trade-offs involves the Iron Triangle of cost, time, and quality (or features). I have written on this often.

The BABOK describes four different approaches to prioritization in its formal description of the technique:

  • Grouping: Assign generic priorities like high, medium, and low, and work through them in order by grouping.
  • Ranking: Rank items in a specific, relative, individual order as opposed to a limited number of generic buckets (as above).
  • Budgeting and Time Boxing: Prioritize based on resources available. This often involves time or money, but other considerations are possible. This is especially germane when the quantity of the governing resource is fixed. This means you do what you can with what you have.
  • Negotiation: Place items or considerations in order based on explicit stakeholder and participant discussion.

Note that decisions on prioritization will sometimes have to be made in the absence of complete information. This is both inevitable and utterly unavoidable. Specific metrics and evaluation criteria should be used when they are available, but when they are not the participants will have to make entrepreneurial judgments on the best way to proceed.

Entrepreneurial judgements should use the best data available, even if it is not perfect. I go into detail about estimation here.

Backlog management involves determining what to do and when to do it. The main current use of the term backlog is probably in the context of Agile projects using Scrum and Kanban techniques as shown in the diagram. However, backlogs can and do occur in every kind of management environment. Statements of work, To-Do lists, punch lists, trouble reports, customer requests, and so on are all examples of backlogs that must be prioritized, managed, and worked off.

Backlog management involves all aspects of how backlog items should be added, tracked, described, prioritized, continually reviewed, worked on, and ultimately removed from the backlog. The purpose of the backlog is to ensure that items are worked on in the order that provides the greatest value first. That said, individual items may not directly provide value themselves, but must be completed to support other items that do.

The BABOK provides the following specific examples of backlog items, but notes that the possibilities are endless.

  • use cases
  • user stories
  • functional requirements
  • non-functional requirements
  • designs
  • customer orders
  • risk items
  • change requests
  • defects
  • planned rework
  • maintenance
  • conducting a presentation
  • completing a document

Items in the backlog may exist at different levels of maturity, granularity, and clarity. The items at the top (or front) of the backlog queue should be clear, defined, granular, and complete so they may be worked on with minimal clarification or additional guidance. Less well-defined items may be added to the bottom (or back) of the queue and will need to be revisited, clarified, broken down, and made ready for work as they migrate toward the beginning of the queue. Backlogs can be continuously reprioritized as conditions, needs, and understandings change, and so they should in no way be thought of as FIFO (First-In / First-Out) queues.

Posted in Tools and methods | Tagged , , | Leave a comment

Interviews

An interview is a process of conducting a conversation with someone to determine what they do, what they need, or how they are reacting to something you’ve given them. The general term elicitation is often used in this context, although elicitation can actually be performed in many contexts, not just interviews.

One interesting thing I have found is that the process of elicitation generally flows in different directions during different phases of an engagement. One the one hand, the analyst is always trying to gain information from a practitioner, user, manager, or customer, but in the early stages of an engagement the interviewer is asking questions and receiving answers based on things the interviewee knows or wants but the interviewer doesn’t know. The information flows primarily from the interviewee to the interviewer. In the latter phases of an engagement, the interviewer asks for the interviewee’s reaction to something the interviewer (or the interviewer’s team) has offered to the interviewee. This flow is shown in the following diagram.

Let’s look at how this might work in each of the standard engagement phases.

Intended Use: The interviewee describes the overall goal(s) of the engagement.

Conceptual Model: The interviewee describes the current operations, job functions, customers, sources, technologies, governing rules, and so on.

Requirements: The interviewee identifies the needs of the organization, solution, and customers.

Design: The interviewer offers one or more proposed designs to the interviewee.

Implementation The interviewer offers implemented solution items to the interviewee.

Test and Acceptance The interviewer offers test results to the interviewee.

This is a gross oversimplification for a number of reasons which I will discuss, but this idea seemed interesting enough to write about and share. I would love to hear your thoughts on whether it makes as much conceptual sense as I have felt it does.

In the end, the whole point of my engagement framework is to inspire and structure robust, two-way communication between solution teams and (external and internal) customers. This means that no communication is only one way, one time. It means that communication is iterative and continuous until all parties agree that understanding is thorough and correct and that an appropriate, value-generating course of action is pursued through every phase.

Let’s look at what might happen in the different phases again with this more complete view in mind. For simplicity and clarity, I will refer to the SMEs, users, managers, interviewees, and so on as the customer. I will refer to the analysis and solution providers as the team.

Intended Use: The intended use may be fairly clear from the beginning, as may be the case when a customer engages a team to provide a known solution. If you want your car fixed, for example, you hire a mechanic to provide a known analysis and solution process. In other cases, however, the problem may not be well defined, and the team can work with the customer to understand and define the problem more accurately and completely. What’s more, ongoing interactions between the customer and team may cause the intended use to be clarified or otherwise modified to add or remove goals. That is, the customer defines the initial intended use, and then the customer and team work together to finalize it.

Conceptual Model: The traditional way of thinking about this involves having the team ask the customer to describe everything they do, to learn the current state. The team then documents the findings and submits the documentation to the customer for review. The customer then provides feedback on what the team got right and wrong, and the process continues iteratively until all parties agree that full and correct understand has been achieved.

That said, the team can (and ideally should) bring a lot of knowledge and experience to this process, and that knowledge and experience can guide the customer to provide a more complete and creative description of the current state. As I have described here, conceptual modeling also involves becoming familiar with tools and solution techniques, market opportunities, and new discoveries which may be leveraged in new solutions.

Requirements: This phase is primarily intended to learn what the customer and all related stakeholders need, but some of the stakeholders in the process involve the team and the prospective solution. That is, depending on the nature of the solution, and especially if the solution is a standard(-ish) one of a type typically provided by the team, the solution, and thus the team, will define some requirements of its own. Moreover, additional requirements may be imposed by governing bodies, (customer and team) organizational practice, and other sources. I have described knowledge teams may (should) bring to an engagement here and here.

Design: The team typically presents a proposed design to the customer, who then provides feedback on its suitability, effectiveness, cost, and other qualities. The customer and team may work together to explore and evaluate multiple solutions. Customers may also solicit designs from multiple teams and then perform their own evaluation and selection. Many combinations and permutations are possible.

Implementation: This one is typically the most straightforward, and classically involves a team implementing a solution for a customer. However, the development, testing, training, and deployment of that solution can be very iterative across many cycles, involve a number of intermediate rollouts, and so on. The implementation and test phases also involve a high degree of iteration and interaction.

Test and Acceptance Testing involves both verification, to see if the solution works as designed, and validation, to see if the solution properly addresses the identified intended use (or organizational need). The team is more likely to oversee the details of verification and the customer will have a larger say in validation.

Many BABOK techniques may be employed during all this review and iteration (think Acceptance and Evaluation Criteria, Observation, Prototyping, Reviews, Item Tracking, and more), but those that involve individual and even small group interactions are primarily interviews. I know I have conducted interviews with almost every kind of worker and stakeholder in a wide variety of organizations and environments all over the world. I have even conducted what are effectively interviews by correspondence (by both email and physical snail mail). The skills I found most helpful are empathy, respect, patience, kindness, clarity, and knowledge of the industry, tools, customers, and methods. Some of those took me far longer to develop than others. Each person will feel more comfortable and excel at different aspects of this process from the beginning, but all practitioners should be mindful of and strive to improve all aspects of this process at all times.

Interviewers should be mindful of all the components of communication shown in the figure above. It takes skill and experience to conduct interviews effectively. However, proceeding in an iterative way that involves review and feedback can help interviewers of every skill level.

Other things to consider are activities that take place before, during, and after an interview. Before activities involve arranging permissions and introductions, setting a time and place, identifying the right people to interview, identifying questions to ask if intending to do a structured interview as opposed to an unstructured one where the line of questioning proceeds ad hoc based on what you learn as you go, sending prepared questions ahead of time, planning how to take notes or record it, and identifying and assuring the required level of confidentiality. During the interview you will want to ensure you remain professional and on track, build rapport, maintain respect, and note any further research of follow-up that may be required. After the interview you’ll want to document what you learned and share it with the interviewee for review and correction, provide your contact information so the interviewee can provide additional information or clarification (or even complaints!), explain how the information will be used (this can be done at the beginning as well), and thank the interviewee for their time and participation.

Interviews have many advantages and disadvantages. Among the more important upsides are the ability to acquire detailed insight from experienced experts and capture a host of non-verbal cues. Some downsides are that they can be difficult to arrange and take a lot of time, especially if many are desired.

Posted in Tools and methods | Tagged , , , | Leave a comment