Bringing Order to Collections and Systems

I’ve recently been putting together a bunch of related thoughts about how I do some parts of systems analysis and design. They involve bringing order to collections of items, data, actions, and ideas. I’ve written about one of the ideas explicitly before and touched on a couple of others. Here I wanted to go into detail.

Collections and Systems

So what are we trying to bring order to? Writing from the point of view of a business analyst or systems analyst it’s clear that this comes up in the course of our work. The collections of stuff we run into are part of the systems we are called upon to analyze, build, and improve.

The raw material of our analyses is gathered through various processes of discovery (identifying the nouns and verbs of what’s going on, see here, here, here, here, and here), and data collection (identifying the adjectives and adverbs that characterize what’s going on, see here, here, and here). As I’ve noted previously, discovery comes first so you know what data you need to collect.

The articles in the links describe many ways analysts learn about the items and processes under study, but I want to add a very important one: brainstorming. This is where people sit together and just vomit out every relevant thought they can about the subject under study. The interesting thing about this technique is that it usually, but not always, involves an associated step of grouping the ideas that seem somehow related.

Finding Commonalities

Here are just a few examples of how people find commonalities.

Frank Bunker and Lillian Moller Gilbreth, whom you may know from the book Cheaper By the Dozen, were pioneers of industrial management and time and motion studies. Through extensive observation they were able to identify a number of simple hand movements that manufacturing workers used in different combinations, and this insight was an important step in making processes more efficient, comfortable, and safe. These basic movements came to be called therbligs, based on reversing the spelling of Frank’s last name.

When I did business process reengineering for FileNet implementations, part of the analysis I learned to do involved identifying similar operations that workers performed in different processes (I’ll call these departments) in the operation. The figure below shows an analysis of all operations identified in a particular department (each of which may have involved up to about fifty workers, as I recall), and the time per worker per day spent on each. The rows identify the unique operations performed by the workers in that department, so all rows have data associated with them. The columns characterized the general type of each operation. Since this list included all types of operations performed by workers in all departments of the overall process, not every column had entries for every department, but every column had entries in the grid for at least one department. It was the information in the columns that allowed us to determine which operations could be automated, and thus how much time and money could be saved. It also allowed us to quantify the new operations that had to be added to the reengineered process, the number of workers and computers required, and their ultimate cost.

I often say that a business or systems analyst is a kind of learning machine, since they have to learn the particulars of the specific things they are analyzing, but they also have to be communication and translation machines. That is, they have to be able to talk to a lot of different of people about a lot of different things, and somehow get them to understand each other. The more the analyst knows about everyone’s job and concerns, the better job the analyst can do.

He is a basic communication model I learned in a leadership school in the Army.

Potentially different techniques can be used to improve each step in the communication process, and this applies whether the communication is in person or electronic. One of the key things is verifying the integrity of each message, and retrying the message as needed until all parties agree that a shared understanding has been achieved. I describe this for electronic communication here (see references to retries), but for people trying to work together through a problem-solving engagement, I refer to the core iterative cycle in my analytic framework, as shown below.

This iterative process helps build understanding, trust, buy-in, and ideally enthusiasm from all participants, and is a form of finding common ground — or commonalities — between people.

Common themes are a hallmark of great literature. While Neal Stephenson’s Cryptonomicon may not be everyone’s idea of great literature, a lot of people think it’s pretty decent science fiction. The story draws parallels between cracking codes in wartime, industrial communications, and interpersonal relationships.

When I was about fourteen I tried to list all the different pieces of Lego I had collected. It wasn’t enough to just list them, though. My precocious, nerdy little self had to try to organize everything into some kind of a taxonomic hierarchy, and that’s where the project hit a snag. Which characteristics would be higher in the hierarchy? Color? Shape? Number and configuration of studs? Height? Something else? I remember that I chose some sort of precedence, but I’m pretty sure I was never really happy with it. Later on I learned about databases, programming, and the ability to sort, display, and analyze flexibly depending on the specific problems at hand. That made me feel a lot better about things.

I didn’t go back, enter all my Lego pieces into an Access database and run a bunch of different reports on them, but these experiences were formative and illuminating. For this conversation, it illustrates that there are many different kinds of commonalities to find. Which ones you find should emerge from the nature of each problem.

The foregoing is an example of there being several possible ways to classify items. By contrast, the Big Five personality traits were identified using only one type of classifying mechanism. As I understand it, they were derived by identifying every possible adjective people can use to describe aspects of a personality, and then a factor analysis was performed on each pair of words (I presume in every combination) to determine the degree to which they were related. When these were analyzed (I don’t profess to know how they did it), the adjectives seemed to fall into five distinct clusters. As someone who’s long been interested in Myers-Briggs personality typing (itself based on the work of Carl Jung), I’ve found this to be quite interesting.

Modularity

As someone who has not only designed and built simulations, but also tools to build simulations, I have a keen appreciation for identifying opportunities to build the smallest number of unique components that can be customized to represent the widest range of entities or processes. I’ve discussed the basic components germane to the creation of discrete-event simulation models here. I documented the creation of my discrete-event simulation demo here, including 91 blog posts. I did something similar when I built a tool to document and calculate coefficients for fluid models.

Being able to identify, design, and implement reusable and reconfigurable components makes systems and architectures much more robust, understandable, documentable, testable, and so on. Said another way, it is a boon to improvement in terms of non-functional requirements.

Rationalization

Finding commonalities can help in the creation of modular designs, but it can bring order to situations in other ways. The best general word I can come up with to describe this process is rationalization. (In a good way, not as Jeff Goldblum’s character invokes the word in the movie The Big Chill.)

I analyzed the product line of controllers and software for HVAC systems to clearly identify what we had, which existing products should be maintained, which features could be added, and what made sense to develop as new products.

As part of this I identified most pieces of communication and functional capability in two dimensions (heirarchal layers on the vertical, protocol or function on the horzontal) on a Visio drawing, and then I created layers for each product the company made, where the relevant elements could be turned on and off as the analysis proceeded. That pointed to ways to make the existing product line more uniform and where it made sense to add new protocols and develop new products entirely. As I review this now, this could have been accomplished a bit more easily and compactly using a spreadsheet.

Sometimes rationalization just means figuring out ways for people to find things. I describe my proposals for monitoring an entire microservices stack across five levels here. This proposal grew out of the need to make the system understandable and manageable, because it was a total fire drill when anything went wrong, with people needing to scatter all over the place to figure out what was happening. This could have been avoided if the information and tools had been organized and placed in a known location where they could be readily accessed and controlled.

I linked to the discrete-event simulation tool I developed as a proof-of-concept (and proof-that-I-could-develop-in-JavaScript) demo above. Each component was different, but I wanted to ensure they implemented similar features in the same way. Here is one discussion that describes a sliver the process I went through (note that the images are scrollable).

Any form of automation can be thought of as a kind of rationalization, because it tends to make each repetition of an action more consistent and repeatable. If structured properly, this can also help reduce errors, build understanding, and improve situational awareness.

As you can see, rationalization can take many forms. It’s just a form of putting things in order — while simultaneously making them consistent — in a way that make sense for each situation.

Final Thoughts

My desire, when I get the chance to make it happen, is to have other people understand that these processes are similar to those of the iterative customer/team feedback cycle I mention above (and often), in that one is always aware of the full process going in and tries to make use of this knowledge throughout any work effort.

It can take the form of getting practitioners to use similar methods and best practices across different activities, can be supported via internal and external communities of practice, and be facilitated by information shared in meetings and through websites and shared communication portals like Confluence and Slack, among many others.

If you can think of any other examples of how you’ve done this, and what did or didn’t work for you, I’d love to hear about it.

Posted in Tools and methods | Tagged , , | Leave a comment

BA Webinar Series 09: How To Unsnarl A Problem When You’re Dropped In the Middle of It

I recently gave this webinar for the Philadelphia IIBA monthly meeting. The slides are here.

Posted in Tools and methods | Tagged , , , , , | Leave a comment

Post University CIS Advisory Board Meeting 2021

Yesterday I participated in yet another of Post University’s Advisory Board meetings for its CIS department (see descriptions of previous years here). The participants included individuals with widely varying experience and perspectives, and it’s always interesting to hear what they have to say. What follows are my own suggestions, based on topics from the agenda, other advisors, and my own observations.

General Topics, Business Analysis, and Soft Skills

I noted that their department is targeted to CIS (computer information systems) and not pure computer science. That means that the context is more about working in different capacities within an organization than going down the rabbit hole of maximum performance and finding the next incremental advance in algorithms and data structures. That suggests that students might benefit from being exposed to a wide variety of concepts during their educations, including core:

  • Algorithms and Data Structures: You gotta start with the basics. This means you typically start with a single language, but it’s a good idea to be exposed to more than one language and some discussions of the applications of and differences between each. When I went to college, must of us started with Pascal, but being exposed to C and assembly language was helpful in beginning to explore the wider world of computing. Since then I’ve worked with or studied close to twenty more, and there are reasons why they all exist. A few of them are and can be expected to remain in widespread use (JavaScript, Node, Java, Python, GO, R, Swift, PHP, C/C++, and C#), some will continue to fade but never completely (FORTRAN, BASIC, Ruby, COBOL), and others may break through or remain in niche uses (Rust, Dart, Opa, Scala, Erlang, Ceylon, Golang, and Hack).
  • Architectural Arrangements: Solutions are built in many contexts, including web-based, microservices, client-server, peer-to-peer, desktop and other standalone, mobile devices, embedded, and IOT. Some will clearly be more germane to CIS graduates expecting to support organizations, but it’s good to be aware of everything that’s out there.
  • Collaboration Tools: Industry tools like SharePoint, Jira, Confluence, Rally, Slack, BaseCamp, Discord, Zoom, Github, email, MS Teams, Azure Devops, and more. Nobody does anything alone any more, especially of any scope, scale, or complexity.
  • Desktop Apps: MS Office apps like Word, Excel, and PowerPoint — or their “free” equivalents from Google, Apple, and others; diagramming tools like Visio, data presentation tools like Tableau and PowerBI, various utilities, and so on.
  • Operating Systems: Unix and Linux (everything from Red Hat to Raspbian for the Raspberry Pi), Windows, Apple desktop, iOS, Android, and others.
  • Management Paradigms: Waterfall, Agile, Scrum, Kanban, SAFe, local vs. remote including offshoring, and work at client sites and consulting.
  • Other: ERP suites, automation and configuration tools (e.g., Jenkins), low-code and BPR platforms (e.g., FileNet, Appian)

Per a discussion of soft skills, I brought up the work I’ve done for my presentations on business analysis, where I often discuss the need for feedback, being thorough, communication models, and generally getting people with different skills and needs to talk to each other. Business analysis is really about the larger context of solving problems as a group, from the point of view of practitioners who can be involved in every phase of any kind of effort. The BABoK (Business Analysis Body of Knowledge), published by the IIBA (International Institute of Business Analysis), describes a wide range of tools, techniques, and general approaches to helping people work together. The sections run as follows:

  • Chapter 1: Introduction (structure of the BABOK)

  • Chapter 2: Key Concepts (basic context of business analysis)

  • Chapters 3-8: Knowledge Areas (the basic flow of what gets done)

  • Chapter 9: Underlying Competencies (Analysis, Behavior, Domain Knowledge, Communication, Interaction, Tools/Tech)

  • Chapter 10: Techniques (50)

  • Chapter 11: Perspectives (Agile, BI, IT, Business Architecture, Process Management)

  • Appendices

…with chapters three to eight generally providing guidelines to work through any kind or project, program, or engagement in a logical order.

I’ve distilled this material down into a more streamlined framework I’ve developed after having worked in many different industries, on efforts of widely varying scope and scale and team sizes, with many different architectural considerations, for many types of target applications, in many kinds of environments all over the world, often while creating, deploying, and using computer simulations. I put this together as a way to give business analysts and project managers a feel for how problems could be effectively addressed and worked through, usually in a team setting where not everyone knows what everyone else is doing. I feel this helps everyone communicate better, be more organized, and produce better work with less wasted effort. The framework is flexible enough to be applied to a variety of working situations.

Thinking back to my own college education, and the things I came to know throughout my career, especially early on, this is material that would have made my life a lot easier, and would have made me a better employee and teammate, if I’d been exposed to it as part of my curriculum. This is despite the overall high quality of the instruction I received. In short, this is very close to what I wish I could learn first if I had it to do over again.

One additional benefit of my efforts is that I’ve come to a much better understanding of the strengths and weaknesses of the processes used by the many organizations I’ve worked for. All had strengths and all had weaknesses, and that raw material gives me a lot of direct experience to work with. (I’ve been meaning to write a separate blog post about this; I need to make it happen.)

As much as I like certifications (which I discuss below), there’s a time to know enough is enough. I often point out that the Wikipedia article for every management or industry-certified approach includes a section for criticisms. They range from the body of knowledge being too diffuse to be useful (e.g., ITIL never gives you specific recommended procedures that lead to specific results), that certs don’t guarantee their holders know anything useful (any Scrum cert), or that they create expensive and wasteful cottage industries of consultants and educators (all management certs). To avoid this, while trying to concretize the material in the BABoK, I present the core engagement steps as a series of iterative loops that involve lots of communication, feedback, and correction. Additionally, the framework builds in the flexibility to iterate between phases to modify and add or delete items in previous or later phases. Like I said above, it’s all about getting people to talk to each other.

I’m especially fond of a talk given by one of the people who helped create the Agile Manifesto. He cheekily suggests that Agile Is Dead, and that what we really need to do is talk to each other, make sure understand each other, make sure we agree we’re doing the right things, and make corrections after so talking. His presentation is on Youtube here, and cannot be recommended highly enough.

Being supportive, engaging, understanding, respectful, and kind — while not wasting people’s time and money — is the best soft skill you can have.

I describe my basic framework here.

I expand on the material in a series of webinars here. The slides for a ninth presentation are here.

The slides for all of my presentations are most easily accessed in the Presentations article here.

AI, Machine Learning, and Data

As an erstwhile mechanical engineer I was never going to be a great computer scientist, even if I probably took as many or more computing courses as any engineer who didn’t get at least a formal minor in the subject. When I took the second level CS course (15-211) in 1983, they covered tree structures, dynamic allocation, and predicate calculus, which was a way of evaluating the truth or falsity of statements written in formal logic. They also talked about how computers could (in theory) understand and describe the world in terms of objects and movements and qualities that could be sensed. I also worked on a year-long senior project and independent study course to create an interactive system to teach engineering students how to solve certain classes of engineering problems. It had a (very rudimentary) natural language interface and maintained situational awareness of where the student was in the process and what ideally needed to be done next. This experience led to a lifelong sensitivity to developments in the (so-called) artificial intelligence field. It similarly contributed to a lifelong interest in education, and its automation and improvement.

One of the main things I’ve learned from watching developments in AI and associated understandings of the brain, nervous system, and sensory systems, is that the more we learn, the more we learn how much there is to learn. The big breakthrough is always thirty years in the future. It was true in 1983, it’s true now in 2021, and I suspect it’ll still be true in 2059.

That said, we do continue to learn and a lot of interesting and useful work is getting done. The question is how much of it to cover in an undergraduate CIS program. I think that students should definitely be given some awareness of the field, and even some exposure to the basics. One of the advisors noted that to researchers in AI can make quite a lot of money right out of school, but that’s for pure computer scientists at top schools (and I know I’m not one of those). Like everything, there are ranges of activities in AI, and some will inevitably be more accessible than others. Data Science and Machine Learning are definitely on the more accessible end of things. For example, a Udemy course on the R programming language covers the following algorithms in order of increasing complexity: Linear Regression, Logistic Regression, K Nearest Neighbors, Decision Trees and Random Forests, Support Vector Machines, K-means Clustering, Natural Language Processing, and Neural Nets. The course doesn’t take the time to go into the algorithms in much detail, and Linear Regression can hardly be considered any kind of AI, but is does show students how to use them. I’m certain that Post University can cover that material and cover it better and more deeply.

I also know that the CIS department includes courses in data structures and databases, so students are already touching on the handling of data, and possibly in large quantities. I think it would be beneficial to include some kind of survey of the contexts in which data are used. I do this in webinar eight of my BA series, which discusses how data may be handled through the course of an engagement, but there are many other contexts including policies (PII, HIPAA), workflow (data pipelines and workflows), display and dashboards (again, Tableau and PowerBI just to start), controversies with Big Tech, collection and conditioning, and automated assistants (Siri, Cortana), among others.

Professional Certifications and Student Portfolios

College diplomas have generally been seen as proof that students can show up regularly, can accomplish certain goals, can reasonably follow directions and cooperate with others, and have a certain amount of intelligence. Grades can communicate another level of information to potential employers, as can impressions made in interviews. But it can be even better to have more visible and specific proof that students have mastered skills and completed projects.

One way is to create and maintain some sort of online presence. This can take many forms, but two main ones are Github repositories, which is most useful for developers, and portfolios, which generally take the form of personal websites. If someone is really clever they can deploy their personal website directly from Github, which would be the best of both worlds. Most of the work on a portfolio site should be original (which could definitely include student projects) or at least non-proprietary. And even then, a portfolio can include quite detailed descriptions of work done for employers. (I do this quite often here on my own site.)

Public websites like LinkedIn and possibly Instagram can host a certain amount of material as well.

Another proof of competency is to earn professionally recognized industry certifications, of which there are many kinds. Certifications may be issued by companies like Microsoft, Google, and Amazon, by industry organizations like IEEE, IIBA, and PMI, or by independent educational companies like Udemy, W3schools, and Codecademy. They may be related to general skills like networking, project management, security, or process improvement, or specific products or technologies like Oracle, AWS, or CISSP.

Like anything, certifications come in ranges of expense, difficulty, time to complete, and general heft. Some have formal work experience requirements. The mid- and top-level IIBA certifications, for example, require applicants to document 3,750 and 7,500 applicable work hours, respectively. Some certifications require that applicants have completed some number of hours from an approved industry education provider or as part of an approved independent study group. In other cases it’s possible to simply read a reference book and pass the exam, though having an awful lot of related experience greatly improves the chances for success. Some certifications will probably be too expensive and take too long to even consider.

The department heads will have to give serious thought to what kinds of certification might best serve new graduates.

Here are a few approaches Post might consider, though I’m sure the department can dream up some other clever approaches.

  • Structure one or more formal courses that lead directly to industry certification. This would include educational hours, practicum, test prep, and collaborative study. The university would have to work out what surcharges, if any, would be needed to cover the cost of exams and materials from third party certifying organizations.
  • Become a certified educational, and perhaps testing provider for some industry organization(s), or become a recognized chapter.
  • Establish and nurture independent communities of practice aimed at earning specific certifications.
  • Give formal school credits for certifications students earn on their own (or in communities of practice described above).
  • Work out student discounts for certifications with certifying bodies.

Conclusion

Thank you as always for allowing me to participate.

Posted in Software | Tagged , , , , , , | Leave a comment

BA Webinar Series 08: Data: How To Get It and How To Use It

Today I gave this webinar for the Tampa IIBA Lunch and Learn Series. The slides are here.

Posted in Tools and methods | Tagged , , , , , , | Leave a comment

BA Webinar Series 07: Testing, aka VV&A

Today I gave this webinar for the Tampa IIBA Lunch and Learn Series. The slides are here.

Posted in Tools and methods | Tagged , , , , , , , , | Leave a comment

BA Webinar Series 06: What BAs Should Know About Software

Today I gave this webinar for the Tampa IIBA Lunch and Learn Series. The slides are here.

Posted in Tools and methods | Tagged , , , , , , , | Leave a comment

BA Webinar Series 05: Simulation

Today I gave this webinar for the Tampa IIBA Lunch and Learn Series. The slides are here.

Posted in Tools and methods | Tagged , , , , , | Leave a comment

BA Webinar Series 04: The Solution

Today I gave this webinar for the Tampa IIBA Lunch and Learn Series. The slides are here.

Posted in Tools and methods | Tagged , , , , , | Leave a comment

BA Webinar Series 03: Unified Theory of Business Analysis

Today I gave this webinar for the Tampa IIBA Lunch and Learn Series. The slides are here.

Posted in Tools and methods | Tagged , , , | Leave a comment

BA Webinar Series 02: Engagement Phases In Detail

Today I gave this webinar for the Tampa IIBA Lunch and Learn Series. The slides are here.

Posted in Tools and methods | Tagged , , , | Leave a comment