R.P. Churchill.com

Focus groups and workshops are separate techniques that involve getting a group of people together for a purpose, but the emphasis in each case is different. A focus group is generally intended to gather people’s reactions to a specific product, service, or situation, while a workshop may be convened for a wider variety of reasons.

Both techniques define an objective, establish a place, time, and procedures to be used, identify participants and roles (sponsor, moderator or facilitator, scribe, SME or other participant, and possibly timekeeper), conduct the discussions and other work, document the results, and follow up on any identified communications and action items. In both cases the organization will (presumably) take subsequent actions based on the insight gained.

Focus groups are often conducted to gauge the desirability or positioning of a new or existing consumer product, but this doesn’t always have to be the case.

Both techniques can gather a lot of information or accomplish other goals in a short period of time, because face-to-face interaction can enhance communication and understanding. Potential downsides include issues of trust, availability, factions, and other issues.

You all know what a glossary is. The BABOK struggles to find much to say about it. It is basically just a list of words and definitions used in the context of the operations being investigated. It is a good idea to create or maintain one when there a many new people in a situation, when you are working in a new industry, when you are using new solution techniques or components, or when engaged in domain knowledge acquisition generally. It is basically an aid to communication, and we know that business analysis is all about communication, especially when trying to translate between users/customers and various technical specialists.

Different terms of art tend to apply in different phases.

• Intended Use: Problem statements are often expressed using broad and understandable language. However, organizational and engagement goals can absolutely refer to specific operations, technologies, or methods from specialized areas of practice. Moreover, the language of business, particularly with regard to financial, legal, and regulatory matters, can employ multiple terms of art.
• Conceptual Model: This is where the language of the operation or solution being built or modified comes into play. It can involve equipment, processes, materials, operations, measurements (Canadian Standard Freeness remains the coolest measurement ever!), and even people (in the form of job descriptions).
• Data: Data management and analysis has a certain vocabulary of its own because it is gathered in so many different ways and has so many different characteristics both singularly and en masse.
• Requirements: Language in this phase tends to use terminology primarily defined elsewhere, although it’s good to be aware of different kinds of requirements (e.g., functional vs. non-functional, business/stakeholder/solution/transition, and so on).
• Design: Work in this phase tends to use language defined elsewhere as well.
• Implementation: Implementation tends to require a wide variety of specialized practitioners, each of which may use specialized verbiage. Software engineers have a large and specialized vocabulary, management frameworks, computer languages, programming paradigms, and the like, but there are many types of practitioners, ranging from pipefitters to graphic artists.
• Test and Acceptance: There are almost limitless ways to verify and validate solutions. Software is just one field and that process has plenty of its own verbiage and even equipment and processes.

Metrics and KPIs indicate how an operation or system is performing relative to trend or a defined goal. They may take the form of point values (ninety-five percent uptime for a manufacturing process), ranges (plus or minus one percent of target), or probabilities (less than fifteen minute wait time, eighty-five percent of the time). They can be measured over time periods of any length.

Many types of values can drive decisions, but what makes something a metric or KPI is how the decision relates to strategic organizational goals. For example, a data point associated with a document or a record may affect whether that item is diverted to one processing operation or another. That data point drives a decision (and potentially thousands of them a day), but that decision does not indicate progress toward a goal or standard. Providing service in an effective and timely manner, reducing waster, hitting quality or financial targets, and meeting or exceeding market standards are all examples of strategic assessments and can be thought of as governing metrics.

Note that values that serve as metrics and KPIs are likely to be derived from the operation of a running process or system, as shown above (at least in part), and not values that describe or govern it, as shown below. Note also that the above figure, at least in this context, refers to a working solution, while the figure below describes the engagement during which it was created or modified.

Indicators may take many forms, but should have many of the following important characteristics (item names from the BABOK, think about how this advice is similar to that for writing user stories):

• Clear: understandable and without ambiguity
• Relevant: germane to the activity
• Economical: obtainable without excess time and effort
• Adequate: embodies enough information to provide effective guidance
• Quantifiable: can be expressed in values that can be validated
• Trustworthy and Credible: deriving from agreed-upon, accurate, authoritative sources

It’s nice when metrics are expressed in terms of simple values, but they can become very complex and synthetic. I worked on one program that tried to improve the “readiness” of large quantities of equipment. The equipment needed a high degree of maintenance and inspection, and was considered “not ready” whenever such work was waiting to be performed or even in progress. As a result, the value of the metric (expressed as a percentage), turned out to be surprisingly low. However, small changes to that value were regarded as highly significant. The important thing is to understand the source, derivation, and context of the metric or KPI, no matter what form it takes. (I could tell you some other stories about that and other programs, but those are discussions for another day. Ask me over a coffee sometime.)

Another issue with KPIs is that they may not capture what is really wanted. For example, I recently encountered a situation where a manager established a goal of having an eighty percent “win rate” for sales encounters. Someone else asked the question of whether the metric would have the desired effect. Going back into history, industrial planners in the former Soviet Union used to set quotas for the production of nails. The problem was that the KPI was expressed in tons of nails, and this was done without consideration of how difficult different kinds of nails were to make. As a result, producers tended to manufacture nails that were easier to produce. Since thicker and wide-headed framing nails are easier to produce than finer and small-headed roofing nails, the shortages of roofing nails were so severe that buildings would be constructed all over the place — except for the roofs. This rendered the buildings unusable and they would just sit and rot in the weather.

The Soviet nail example is meant to illustrate the advantages of the price system, where the (presumably higher) price of roofing nails incentivizes producers to manufacture what is needed.

I don’t know if we always have to go as far as saying Goodhart’s Law is always applicable, but one has to be mindful of what is being measured and what it means.

Returning the “win rate” example I started with, is the real goal to ensure that sales teams get wins at a certain rate, or is it to make the most sales? Or is it to make the most sales per a given amount of effort? Or is it to bring in the most money? One possible problem with targeting the win rate alone is that tales teams may game who they try to sell to, or they may do inappropriate things (like reduce the price to buy sales, thus lowering the brand’s value). None of those are desirable.

Metrics and KPIs should be revisited and reviewed over time to ensure they represent what is wanted, and that they incentivize the desired actions. I often mention the need to be thorough, and to look at problems from multiple points of view. As always, different people with different perspectives should be consulted where it makes sense. I try to illustrate this by including the image below in many of the talks I give.

Prioritization and backlog management are two sides of the same coin. As described in the BABOK, an important difference is that backlog management is a more specific case within the broader topic of prioritization.

The BABOK discusses prioritization in terms of business analysis information. Backlog management typically involves tasks or work items. As a practical matter, the criteria on which prioritization is carried out in both cases is the same. Considerations involve value, risk, difficulty of implementation, dependencies, and so on.

It seems to me that prioritization can involve qualities of a design or approach, determining which customers to serve first, and the order in which tasks should be worked on. While backlog items typically involve defined collections of work to be done, they are ultimately driven by the same, more general considerations.

A very important consideration in prioritization involves recognizing and making trade-offs of various kinds. Many types of trade-offs are possible, and all involve the economic way of thinking. Economics, after all, is best defined as the study of making choices under conditions of scarcity, and this involves the recognition that we always live and work under conditions of scarcity. One discussion on trade-offs is embedded in a webinar I gave on What Business Analysts Should Know About Software.

While this specific talk involved software, you can imagine that different trade-offs will be germane to other potential products and services. Considering the iPhone, for example, Apple could not produce them in every possible color. Instead, different color options (moving in a direction from conservative to increasingly bold over time) have been released only slowly, to where there remain only six color options after fifteen years. That is, the designers and managers at Apple had to look at the costs and practicality of producing devices in different colors.

Computing
• execution speed
• response time
• memory usage
• disk usage
• clarity
• modularity
• maintainability
• reusability
• abstraction
• language choice(s)
• protocols
• scalability
• uptime
• security

People and Methods
• developers
• managers
• data collectors
• analysts
• testers
• users
• customers
• speed of development
• reduction of rework
• popularity of languages and tools
• industry trends
• larger societal trends

Cost
• capital vs. ongoing
• fixed vs. variable
• licensing
• hardware
• third-party
• maintenance and support
• upgradability
• modifiability

Another classic discussion of trade-offs involves the Iron Triangle of cost, time, and quality (or features). I have written on this often.

The BABOK describes four different approaches to prioritization in its formal description of the technique:

• Grouping: Assign generic priorities like high, medium, and low, and work through them in order by grouping.
• Ranking: Rank items in a specific, relative, individual order as opposed to a limited number of generic buckets (as above).
• Budgeting and Time Boxing: Prioritize based on resources available. This often involves time or money, but other considerations are possible. This is especially germane when the quantity of the governing resource is fixed. This means you do what you can with what you have.
• Negotiation: Place items or considerations in order based on explicit stakeholder and participant discussion.

Note that decisions on prioritization will sometimes have to be made in the absence of complete information. This is both inevitable and utterly unavoidable. Specific metrics and evaluation criteria should be used when they are available, but when they are not the participants will have to make entrepreneurial judgments on the best way to proceed.

Entrepreneurial judgements should use the best data available, even if it is not perfect. I go into detail about estimation here.

Backlog management involves determining what to do and when to do it. The main current use of the term backlog is probably in the context of Agile projects using Scrum and Kanban techniques as shown in the diagram. However, backlogs can and do occur in every kind of management environment. Statements of work, To-Do lists, punch lists, trouble reports, customer requests, and so on are all examples of backlogs that must be prioritized, managed, and worked off.

Backlog management involves all aspects of how backlog items should be added, tracked, described, prioritized, continually reviewed, worked on, and ultimately removed from the backlog. The purpose of the backlog is to ensure that items are worked on in the order that provides the greatest value first. That said, individual items may not directly provide value themselves, but must be completed to support other items that do.

The BABOK provides the following specific examples of backlog items, but notes that the possibilities are endless.

• use cases
• user stories
• functional requirements
• non-functional requirements
• designs
• customer orders
• risk items
• change requests
• defects
• planned rework
• maintenance
• conducting a presentation
• completing a document

Items in the backlog may exist at different levels of maturity, granularity, and clarity. The items at the top (or front) of the backlog queue should be clear, defined, granular, and complete so they may be worked on with minimal clarification or additional guidance. Less well-defined items may be added to the bottom (or back) of the queue and will need to be revisited, clarified, broken down, and made ready for work as they migrate toward the beginning of the queue. Backlogs can be continuously reprioritized as conditions, needs, and understandings change, and so they should in no way be thought of as FIFO (First-In / First-Out) queues.